Skip to main content

Remove Duplicates And Keep Most Recent Records in Power BI

 

A week ago in my last training, I was demonstrating how to get rid of duplicate records when making a dimension table.  One of the people in attendance asked is there a way to get rid of duplicates but keep the most recent duplicate record based on a date column.  I thought, “Of Course!”.  All we need to do is use the Query editor, do a sort on the date column, and then get rid of duplicates.  Unbeknownst to me, it didn’t work.  It kept the oldest record and not the newest.  



The reason being after doing some research is that the Power Query editor will make determinations if it can do some of your steps in a different order and still give you the same result while producing the quickest data load.  Normally, that is what an ideal Power BI user wants, but in this case, we need the sort to happen before the remove duplicates step.  Thus, after research comes in the function Table.Buffer to modify our sort code.  


Take a look at my video for a further explanation and demonstration of how to use this function to your advantage.




Comments

  1. James ZicrovJuly 16, 2021 at 1:02 AM

    Thank you so much for providing such a unique and descriptive analysis of Power BI and REST API.

    Powerbi Read Soap

    ReplyDelete

Post a Comment

Popular posts from this blog

The Power of CALCULATE in Power BI

Last week in my blog and video I discussed how Power BI could have made my life as a teacher much easier.  It is not that Power BI can do things that I could not on my own.  It is the fact that Power BI could have saved me so much time.   This led me to some thoughts and questions.   How could I have impacted the students more in the classroom with this extra time?  Would teachers who were not analyzing student data due to time constraints now start?  How can I make the data tell my students a story that gives them an accurate picture of their progress?   That last question is what leads to this week’s blog.   Many times the metrics (which is just a fancy word for "numbers") of what we want to show are easy to do by hand with a basic math background.  Although easy, it can be very time consuming when dealing with different groups of data: like 6 periods of students instead of 1 period. When we want to do these calculations quickl...
7 comments
Read more

Relating "Related Tables" to Baseball because I Miss Sports

I miss sports. In particular, I miss baseball. Between learning more Power BI functions and the ins-and-outs of DAX, I've turned to Netflix to fill the deep caverns left in my soul since baseball season has been postponed. And as a result, I've thought more about tigers and big cats more than I ever have in my life. I know ALL about Carol Baskins and am fully on board for a spin-off centering on locating her lost husband. I've googled "is it really legal to own a tiger in a residential area?" Without baseball in April, I am barely hanging in there (kinda like Joe Exotic's eyebrow ring). So, I am filling the sports-sized hole by using baseball stats in Power BI to demonstrate pulling data from multiple tables and consolidating it into one table.  Some of the data we want to consolidate also has to have some aggregations (which is fancy for "calculations") performed on it.  In this demo I will attempt to break down what is really going on ...
Post a Comment
Read more

Power BI: Personalize Visuals Feature

    Have you used the personalize visuals feature yet in Power BI?   If not, then you could be missing out.   The Personalize Visuals feature allows anyone you are sharing your with after published to the Power BI service to make changes to your visuals.   Have no fear though!   When they change the visuals it will not override your original report.   This feature is great if you have ever had to respond to a request from your shared users that they would like the report to be modified to their liking.   With this simple feature, the end-user can change the visualization type, change what measures or fields that are brought into the visual, and then simply make a personal bookmark after they make the changes.   Again, this will not change the initial report you published and that others have access to.   This will only make a new view of the report for the user who personalized the visual.   If you decide to use this feature...
2 comments
Read more