Skip to main content

Remove Duplicates And Keep Most Recent Records in Power BI

 

A week ago in my last training, I was demonstrating how to get rid of duplicate records when making a dimension table.  One of the people in attendance asked is there a way to get rid of duplicates but keep the most recent duplicate record based on a date column.  I thought, “Of Course!”.  All we need to do is use the Query editor, do a sort on the date column, and then get rid of duplicates.  Unbeknownst to me, it didn’t work.  It kept the oldest record and not the newest.  



The reason being after doing some research is that the Power Query editor will make determinations if it can do some of your steps in a different order and still give you the same result while producing the quickest data load.  Normally, that is what an ideal Power BI user wants, but in this case, we need the sort to happen before the remove duplicates step.  Thus, after research comes in the function Table.Buffer to modify our sort code.  


Take a look at my video for a further explanation and demonstration of how to use this function to your advantage.




Comments

  1. James ZicrovJuly 16, 2021 at 1:02 AM

    Thank you so much for providing such a unique and descriptive analysis of Power BI and REST API.

    Powerbi Read Soap

    ReplyDelete

Post a Comment

Popular posts from this blog

The Power of CALCULATE in Power BI

Last week in my blog and video I discussed how Power BI could have made my life as a teacher much easier.  It is not that Power BI can do things that I could not on my own.  It is the fact that Power BI could have saved me so much time.   This led me to some thoughts and questions.   How could I have impacted the students more in the classroom with this extra time?  Would teachers who were not analyzing student data due to time constraints now start?  How can I make the data tell my students a story that gives them an accurate picture of their progress?   That last question is what leads to this week’s blog.   Many times the metrics (which is just a fancy word for "numbers") of what we want to show are easy to do by hand with a basic math background.  Although easy, it can be very time consuming when dealing with different groups of data: like 6 periods of students instead of 1 period. When we want to do these calculations quickl...
7 comments
Read more

The Teacher Becomes the Student

My wife asked how my first day was going so I sent her this selfie. Only a picture could capture it.  First, let me tell you that this won’t be your typical Power BI or tech blog that you usually see. Let me set the scene. It was a year and a half ago when I got lucky enough to be introduced to Brian Knight, the CEO of Pragmatic Works over lunch. He had heard from a mutual friend that I was looking to make a career move after teaching Algebra for the past 15 years. Brian started telling me about the company and all the different facets it encompasses. From training, sales, consulting, and app development. Mind you at this time I had never heard of Microsoft Power BI, SQL Server, Microsoft Azure, or have any experience with coding. He told me an opportunity at his company may present itself in the future where I could transfer over my teaching skills into the training side of the above-mentioned programs. I immediately got to work with their On-Demand Learning system to ...
3 comments
Read more

Power BI Makes APPs, too? A Quick Guide to Understanding that an APP is like a CONTAINER

POWER BI can make apps too?   Wait, then what is the point of Microsoft Power Apps?   Well, that’s where there can be confusion and I see it often during my training sessions.   An app in Power BI is COMPLETELY AND UTTERLY DIFFERENT from an app in Microsoft Power Apps.     According to Microsoft a Power BI App is, “ An  app  is a Power BI content type that combines related dashboards and reports, all in one place. An app can have one or more dashboards and one or more reports, all bundled together.”  Notice how the word app has been italicized.  I didn’t format it that way.  I believe "app" is italicized in their documentation because the semantics can be confusing and intermingled. So let me give you my brief overview of a Power BI app.   Next week’s blog will dive into deeper details and have a video demonstration as well. I like to think of an app as a container where you place all the dashboards, reports, and workbooks you w...
Post a Comment
Read more