Best Practice To Trim Before Removing Duplicates or Merging In Power Query Editor

In last week’s blog, I wrote and did a video about how to remove duplicate records and keep the most recent entry as long as a date column was part of the data source. I came across the scenario while giving training on Power BI with my company Pragmatic Works. See the video below:

This week, while doing another two-day training I came across a different scenario from a follow-up conversation from day 1. I had explained how to remove duplicate records and one of the students started working on a Power BI project she has for her company. On day 2 the student informed me that her remove duplicates step was not working. I said that is odd and I asked to see the data. In one of her table visuals, I could see that it appeared that a few of the records had duplicates based on the name column. After further investigation though, we figured out the culprit.

She had done all the steps correctly, but it was a data integrity issue. In her data source, the person in charge of entering the data had mistakenly typed a space for a few of the records before entering the name value. Thus, when Power BI looked for a duplicate in the name column it did its job correctly because “Jamie” is unique compared to “ Jamie”. That leading white space was the culprit. Not only did it affect her Remove Duplicates step, but it would also have affected a merging of two queries if she had included that as part of her report.

For this week’s video, I want to show how to use the Trim function in the Power Query Editor as a best practice to avoid these unwanted outcomes for any future merges or removing of duplicates.

Comments

The Power of CALCULATE in Power BI

Last week in my blog and video I discussed how Power BI could have made my life as a teacher much easier. It is not that Power BI can do things that I could not on my own. It is the fact that Power BI could have saved me so much time. This led me to some thoughts and questions. How could I have impacted the students more in the classroom with this extra time? Would teachers who were not analyzing student data due to time constraints now start? How can I make the data tell my students a story that gives them an accurate picture of their progress? That last question is what leads to this week’s blog. Many times the metrics (which is just a fancy word for "numbers") of what we want to show are easy to do by hand with a basic math background. Although easy, it can be very time consuming when dealing with different groups of data: like 6 periods of students instead of 1 period. When we want to do these calculations quickl...

The Teacher Becomes the Student

My wife asked how my first day was going so I sent her this selfie. Only a picture could capture it. First, let me tell you that this won’t be your typical Power BI or tech blog that you usually see. Let me set the scene. It was a year and a half ago when I got lucky enough to be introduced to Brian Knight, the CEO of Pragmatic Works over lunch. He had heard from a mutual friend that I was looking to make a career move after teaching Algebra for the past 15 years. Brian started telling me about the company and all the different facets it encompasses. From training, sales, consulting, and app development. Mind you at this time I had never heard of Microsoft Power BI, SQL Server, Microsoft Azure, or have any experience with coding. He told me an opportunity at his company may present itself in the future where I could transfer over my teaching skills into the training side of the above-mentioned programs. I immediately got to work with their On-Demand Learning system to ...

Power BI Makes APPs, too? A Quick Guide to Understanding that an APP is like a CONTAINER

POWER BI can make apps too? Wait, then what is the point of Microsoft Power Apps? Well, that’s where there can be confusion and I see it often during my training sessions. An app in Power BI is COMPLETELY AND UTTERLY DIFFERENT from an app in Microsoft Power Apps. According to Microsoft a Power BI App is, “ An app is a Power BI content type that combines related dashboards and reports, all in one place. An app can have one or more dashboards and one or more reports, all bundled together.” Notice how the word app has been italicized. I didn’t format it that way. I believe "app" is italicized in their documentation because the semantics can be confusing and intermingled. So let me give you my brief overview of a Power BI app. Next week’s blog will dive into deeper details and have a video demonstration as well. I like to think of an app as a container where you place all the dashboards, reports, and workbooks you w...

The Algebra Teacher Powers to B.I.

Search This Blog