Merging Duplicates

Hello, and welcome to our webinar, Merging Duplicates. This is part of our Database Maintenance series. My name is Donna Mitchell. And I’m really happy to be here to help you with this webinar today.

With that Read More

Hello, and welcome to our webinar, Merging Duplicates. This is part of our Database Maintenance series. My name is Donna Mitchell. And I’m really happy to be here to help you with this webinar today.

With that without clean, accurate, complete data, all the whiz bang features in the world CRM integrations, return on investment reports, automatic campaigns, and so on are no good. All this great functionality revolves around the data in your database. As we know, garbage in, garbage out. So if we’re putting great information in and keeping our database clean, it’s going to be very helpful to be able to get the data out the way we want. That’s why we’re going to talk about merging duplicates today. So in today’s session, our goal is are no duplicates in our system, we don’t want to see any duplicates. So what we’re going to do is we’re going to show you the different options you have for merging records in your database. And we’re going to do a demo on merging duplicate records. Now, the contents of this webinar may contain features and fields that are different than your own. Here are some examples of some duplicates. So if we take a look at this first example, you’ll see that this is really truly a duplicate, we can see that the names address, phone numbers are the same. And for some reason they got in there, there are duplicates, so we would be able to merge those records. And we wonder why are there two, we would need to repair that. In this next example, you’ll see here that there are some information that’s a little bit different. From one record to the other. You’ll see the names are, there’s Joe and Joseph, you’ll see the address is a little bit different. But it’s it has the same numbers. So we would probably merge these two, because they look like they truly should be merged together. In this particular example, we’ll see that we have an organization record. And then we also have an individual record for Sara, who is the contact at this organization. But in this case, it may actually be a duplicate. Now there are situations where the person that is the contact that your organization is also an individual donor, that could be the case in this particular case. Now, if that’s not the case, so we’re actually going to be able to merge those records together. And finally, you may encounter these situations where you don’t really have any common denominators to do a merge to compare the records, there’s really nothing here, that would tell me that these are duplicate records, this is something that I know. And oftentimes, because you know your donors, so very well, you would be able to realize that, oh, Fred, and Roberta actually are in the same household, they belong in this one record. So you would be able to merge that. Now. That is, these are some of the scenarios that you might encounter, we’re going to show you how to create duplicates, we’re gonna go over the steps and things you need to do. And then we’re going to do some demos to show you how to merge records. So the first step in doing a merging duplicates, and DonorPerfect would be to create a backup. And we often recommend that you do a backup. Whenever you’re doing anything major to your system, like merging duplicates, maybe global updates, importing using the screen designer and things like that, we always recommend that you do a backup. So a backup really is the storage of the data that is otherwise gone, or permanently changed. So that’s one of the reasons why we want to do a backup. If we’re making a big change, where we are deleting records or changing information permanently, we want to be able to undo that process if we make a mistake. And that’s exactly what backing up your system will do. Now, we need to make sure that you understand that this was a user initiated backup, we do back you up every night. It’s automatically stored, backed up every night and stored on our servers. We hold this information for 30 days. Now, if you need us to restore one of our automatic backups, it will cost you money to do so. So we always recommend again that when you’re doing major things to your system, go ahead and do a backup. So we’re going to go into DonorPerfect and show you how simple it is to do a backup in your system. And there’s DonorPerfect, and we’re going to go to the menu ribbon to utilities. And you’ll see the first option Backup and Restore and a good clue here is if you’re doing anything in this particular area, the There are good reasons to do backups, global updates, import and merging duplicates. So let’s go into Backup and Restore. And you will see what we have here. Now we do tell you, we back you up, and it gives you a little more information as to why you should backup. But you’ll see here you have three backups, you will have three backups on your system at any given time, you are able to create a new backup and it will overwrite the oldest backup. So that’s how it works. It’s very simple. We click on Create new backup, it will warn you that if you have three, it’s going to overwrite the oldest, we want to click on OK. And you just wait and that quickly your system is backed up. Now clearly, if you have a lot more records, it will take a little bit longer. But it really is not a long process. So we click on OK. And now you will see we have our new backup with our new record count. If we do these changes, and for some reason, we don’t want that to happen anymore, we can come back and click on Restore. And this will restore the system to the way it was prior to your backup. So that is your Undo button, that’s your safety net, make sure that you do a backup. We also have external external data assurance. And that’s available for an additional fee to have the ability to download the automatic backups that we create for you. So you would reach out to your account manager about that. Now, merging duplicates. So when we’re merging duplicates, we have some different things that we need to talk to you about with regard to that. So the ways that we search for duplicates is in pairs. Okay, when we go to do our search, you’re going to see that the possible duplicates are matched up in pairs. And that’s how we’re going to work with them. We’re going to also talk about the main entry screen address from the duplicate record will be added to the final combined record as an alternate address. And that’s one of the beautiful things about merging duplicates, you don’t lose any of your data. So when you’re merging two records, just realize that you’re actually getting rid of one record. So the good news is, the address from the record that we’re getting rid of becomes an alternate address on the final combined record. Now transactional records from two duplicate records, such as your gifts, pledges, contacts, other info, alternate addresses, all of that data will be merged together. Again, in that final combined record, all of that information will be saved. flags in the deleted record will be added to the final combined record if they are not already there. So if the records are getting rid of its flag as a volunteer, and you merge it into a record that doesn’t have the flag, the flag will automatically appear in the combined final record.

Alright, so when we’re going through merging duplicates, the first thing we have to do is search for duplicates. Now, the most important thing to remember is you have matching criteria with which to do this, and you’re going to look at the duplicate report. Once we get in there, you’ll see there’s a duplicate report that will show the duplicates, and then we can combine manually combining manually is an option that we have if we don’t have that common denominator, okay, so we would be able to select the records that we know are duplicates, and manually combined them, we’ll show you how to do that. We also have an option called Auto D dupe, auto D dupe allows the system to scan the records, and it will merge those records that are an exact match. And when we get into merging duplicates, I will show you there’s a little information bubble that you want to take a look at, it will only merge exact duplicates. So we’ll talk about that. Now. Let’s move on and show you how easily you can delete a record from DonorPerfect. We don’t typically recommend doing this but you can go to the search screen. Okay, let’s just do a search. I’m going to do our friend the munchies oh now, I’m not going to delete this record, but I’m going to show you that it is possible to do so and you can click on this Delete button and that will delete the record that you don’t want. Now, our recommendation is not to do that. But if you feel compelled to do that, you definitely want to export as much data from that record as you possibly can. before deleting the record, we would rather you go through the duplicate process This and merge these duplicates, so that you keep all the data, right. So that is an option. Again, not highly recommended, but we wanted you to know that that is possible. All right, so let’s talk about the duplicate report. So the first thing we have to do is find the records and choose how we’re going to compare the data. So here is the screen that you’re going to encounter. When you select Merge duplicates, you’re going to choose how to compare for your duplicates. Now, what you’ll see here are the you will see here, we have matching criteria. So it’s going to compare records on the first name, the last name, I’m sorry, last name, first name, address, and zip. And it’s going to do so on the number of characters that you see here. So it’s going to match the records based on the first 10 characters of the last name, the first eight characters of the first name, the first eight characters of the address, and the first five characters of the zip code. Now you have the ability to you have the ability to add or remove criteria, you can also raise or lower the number of characters to match in here. So you have a lot of flexibility. And this is going to be helpful, again, helping you to find those records that we wouldn’t normally find if we use all of these. And normally, what we tell you to do is select them all run your report, see how many you have clean those up, if you don’t have any, where you have very few, then you begin to remove criteria and or lowering the number of characters to compare. All right. Now, when you run the report, based on the matching criteria you’ve selected, you will see the names that will come up that are possible duplicates. Now we have a wonderful tool here that allows you to Mark records as not duplicates. There are times when records will show up as possible duplicates, but they’re not really duplicates. For example, this is Mr. John Howard Jr. This is Mr. John Howard, this is the senior John Howard. They’re at the same address. So in this particular case, they’re both giving separately. And I know that because I know my donors, I’m going to mark them as not duplicates, they will never show up as duplicates of each other again. So again, this is a really wonderful tool for those records, that actually should not be duplicates. Now this record, Kelly’s record, 199 and 207 are duplicates. So I would go through the process of merging those records together. All right. So let’s go in to DonorPerfect, we’re going to go back to utilities. But this time, we’re going to go to merge duplicates. Okay, I’m going to leave this as it is. And we’re just going to assume what we see is going to they’re not actually duplicates. So I’m just going to remove the zip code and the first name, and I’m going to show the report. So at the bottom left, you’re going to click on Show Report. And there you will see your records. Now, you have Aaron Johnson and Donna Johnson. Now clearly, they’re not duplicates. In this particular case, they have the same address, but they’re in different cities. So I could mark them as not duplicates. And again, they will never show up as duplicates of each other again. So let’s go do that. I click on OK. And now those records will no longer show up as duplicates again.

All right. Now we have some different ways that you’re going to be able to combine. One of them is the one that I think a lot of folks might find helpful is combining manually. You don’t there’s no similarity in the records. There’s no similar address, there are no common denominators. In this case, you get to select each of the records and be able to merge those manually, regardless of the fact that you’re doing it manually. All of that data will still move to the one record. So it’ll combine all the transactional data, gifts, pledges, contacts, all of those things. So we would be able to find those records simply by looking them up because we know they’re duplicates. And then we would be able to combine them manually. Okay. Now I’m going to show you how to do that in a little bit. And we’re going to talk about how the rest of our merges work. Now, combining duplicate records, we have two ways that we can combine the records, we have a simple combine the simple combined gives us a very small amount of information to go on. To be able to merge the records, the advanced combine is a little more flexible in that it gives us a larger amount of information and more flexibility to move the fields to the final record that we want. So you do have the option of selecting which fields in the record that you’re getting rid of that you want to add to the new combined record, it’s a really great way to see all of our donors data on each of the records, and then choose the fields that you want to be able to combine. Again, regardless of the fact that we’re combining these records, all that transactional data will be moved to the new merged record. So what I want to show you now are, is the simple combine, the simple combine is just that it’s very, very simple. You will see here that we have arrows pointing to the record to be saved. So if you take a look here, I’m just going to get my marker here, you’ll see here we have a radio button, pointing to this record. And if we select this record, then this is the one that we’re going to keep. If we select the top radio button, then that is the one that we’re going to keep. So we have to decide what information we want. And again, all transactional data will be moved to the record that we keep. So let me explain how the simple combine works. Because we don’t really have the ability to edit the record. When we’re doing this simple combined, we have to understand what happens when there are data in some of the fields and no data in others. So in this first scenario, this is showing you that we’re going to keep the left record record 1012. If I keep that, you’re going to see that if there are empty fields. Okay, so there’s 1012 1012, we’re going to keep 1012. And you’ll see here that in record 1012 that I’m going to keep the professional title and the optional line fields are blank. However, in the record that I want to merge it with the professional title and the optional line are populated. So what happens when we merge these records, we will actually see that it will populate those empty fields. So if the record that you’re merging to has empty fields, and the record that you’re merging from has those fields populated, they will populate the resulting record. So that is scenario one. Let’s look at scenario two. So in this case, we have the same records, right, we’re doing 1012. But this time, we’re merging 1012 into this record. And you will notice again, that the professional title is blank, the optional line is blank. And we come down to the resulting record. And now you will see because these fields were blank, they’re going to populate the professional title and optional line, giving us this information in the final resulting record. Okay, so again, it fields are blank in the record that we’re pulling from, and the fields are populated in the record we’re going to it will keep that populated information. Now, scenario three, let’s see what happens here. So we’re merging 1012 into 101327. Now, if you take a look, you’ll see the salutation reads John and you greet. And here it says John, the professional title says actor and the professional title here says actor and of course we have our optional line with records name. Now when we merge these records, because we’re merging into this one, you’ll notice that the salutation remains the same. So if there are data in the field that you’re merging into, it will not overwrite with the information from the record you’re pulling from. However, it will fill in anything that is blank. So that is important to remember, we’re merging this record into this record. It does not keep John India as the salutation because it’s already populated as John. So you would have to go in and manually update this record. If you’re doing the simple combine Now we’re going to talk about the advanced combine. So the advanced combine, again gives us the ability to choose the data that we want to end up in the resulting record. And this is a really great way for you to get the record exactly the way you want. Now, you’ll notice here that regardless of what we do, even in the simple combine, we have the ability to edit the record in don’t perfect, but we’re going to just look at moving the fields around the way we want. So what you’ll see here, in this case, we have Mr. John Snow in the record. So column one is record one, column two is record two, and column three is going to be the resulting record based on the fields that I select here. Okay, so you’ll see I selected John Snow, Mr. John Snow, and it reads Mr. John Snow. Here we have the salutation John and your greet actor is the professional title. And the optional line with your greets name. Now it’s not here. But what I did was select the radio buttons next to these fields. And now you will see that they populate this final record. So now my final record is as complete as can be, I can have everything in here that I want. And it’s going to display and that’s what my resulting record is going to look like. All right, so now it’s time for a demonstration. So let’s go. The first thing we’re going to do is we’re going to go into utilities, and Backup and Restore. Now I know we’ve done a backup, but this is what you have to do first. So I wanted to show you again, to make sure that you’re doing that. There’s the backup and restore. So we just click on backup, and we’re ready to go. We’re going to go back to utilities and merge duplicates. All right, so the first thing I want to show you and we didn’t talk about auto D dupe, but auto D dupe is an option too. So let’s just take a look at this page. You’ll see here, on the left, we have all of their fields that are populated by default. So we’re going to search these are my search matching criteria, and it’s going to use the number of characters in each field. Again, you can add and remove matching criteria, you can also raise or lower or lower the number of characters to find your records. Now at the bottom of the screen, you’ll see it has a settings for the manual combined, which we’re going to do in a moment. When I go to do a manual combine, I can choose whether I want to do it simple, or the advanced option. Then we have auto D dupe. And again, auto D dupe is going to I’m going to click on this little question mark here. And you will see let’s move this up so you can read it. Otto de doop has the system scan for duplicates, however, they will only merge exact matches only exact matches. So in this scenario, as you can see, we have more ksql 123. Wells Road, same record, these records would merge because they’re an exact match. However, if we go and we have a Marcus Segal address, same address as Mark Segal, it will not merge because it is not an exact match. So when you’re doing auto de doop, you need to think about that. And you will have the ability to choose if you want to keep the older or newer records. So let’s see how auto d do works. Then we’ll do an advanced combine a manual combine, and a simple combine. So let’s go in, I’m going to select Auto D dupe.

And I’m going to click one Show Report. Now when I click on Show Report, it’s going to show me all of the records that are our exact matches. And here we see Jamie Smith exact match, it’s going to tell you the total number of duplicates based on your parameters. And right now we have one, it’ll tell you the amount of time it thinks it’s going to take to merge. But you also have the ability to determine which direction you want it to merge older to newer or newer to older. Now this is a personal preference of your organization. Whatever you choose, I happen to be a fan of merging newer to older, I want the older records to be the first one. So I’m going to select newer to older and I’m going to click Auto combine selected records now, warning it comes up it should say that WARNING WARNING WARNING you are about to combine records, make sure that you are clear about what you’re doing. And then you’re going to be able to combine them and so now record number three 315 will not be there anymore, but all that data will merge to the record. Okay, so it says Please wait while the duplicates are being merged, do not stop, do not refresh or close the browser window, it tells you here that all duplicates have been combined, please press the button below to return to the merge duplicates screen. So that was very quick, very, very quick. Now clearly, if you had a lot more, it would take a little bit longer, but you want to make sure that you don’t interfere with the process at all. All right, we’re going to return to merge duplicates. And the first one we’re going to do is the manual combine. So we have two records that we know are duplicates, but they have no common denominators. So we want to be able to merge them together. But we can’t use any of the matching criteria, because there are none. All right, so what we’re going to do is come down to the settings for manual combine. And we’re going to choose how we want to combine them, I’m going to do the advanced combine, because again, this gives me more flexibility and choosing the fields that I want to be in the resulting record. So let’s select that advanced combine, and then we’re going to click the button that says combine manually. Okay, when I get here, I’m going to look up the donor IDs of the records that I want. So I’m going to go to look up name. And of course, you can use any of the fields that you want. I just happen to know the donor IDs. So I’m going to go to my manual combine. And I want record for 55. Okay, search, there is Jennifer Holland. Alright, 229 81 Maple, Glen Road, Horsham, I’m going to select that. Okay, I’m going to click lookup name here. And I’m going to choose Record 3923 92. And we’re going to search. And there’s Mr. Henry Hughes. So let’s take a look at these, I’m going to select that. And as soon as I do that, it’s going to open up to the advanced merge. This is where I can select the fields that I want to be in the resulting record. Now if you take a look at the top of each column, you are able to go in and edit the records in DonorPerfect if you needed or wanted to. So that gives you the ability to do that. Now what you’ll see here in column one is record one, column two is record two. And again, column three is going to be the resulting record when I go to combine these. Now, what’s really nice about this is I do have the ability to edit. So let’s see what we can do here. So we have Jennifer Holland, and Henry Hughes. The addresses are just about this is 2981. Maple Glen road, this is 2891. Glen road, it should be maple. Glen. So what I’m going to do is choose that option. So it’s going to change it to 2981 Maple Glen road. Now here in my salutation, I have I simply have Henry, because I can do an edit in the simple combine. I’m going to type in and Jennifer. Okay, so we’re going to make it that there we are. Now if I scroll down a little bit more, I see all of this information. That’s all good. We have the email, there’s no email here, we’ll see. This is the record, it was created date. But regardless, it’s going to combine the gift totals now. Excuse me, this may have been an in kind gift, a $0 gift is typically an in kind gift. But regardless of that, you will see that it combines those gifts together. So I have three gifts totaling the $75. Now I can scroll down a little bit more, see what else is going on here. Here’s the full name, excuse me full name. And this will change once we merge the records. And then we see the full name fields. And again, all of this will be updated and changed for us. All right, and again, we have all the way down. Again, typically you don’t see things down here, too much that needs to be moved. But if you do, you’re certainly welcome to move them. And now we’re going to click on combine. So we’re going to end up keeping record 392 And it’s going to fill in with this information. The other thing I want to do, because Jennifer has a different last name. I want to use this optional line to indicate that so I’m going to put Jennifer Holland in here. So it should be Holland There we are. So because as they are two people in the same household with a different last name, we want to put Jennifer’s name in the optional line. So now this will update the record to include the optional line. Okay, let’s click on combine. Perfect. Now we’re going to click here to go back to our merged list. And now we’re going to go back and type in 392. Once again, and we’re going to take a look at the updated record. And it updated exactly the way we asked it to. And there you will see, and reuse, and reinvent Jennifer, Jennifer is now in the option line with the updated address. Okay, and of course, the number of gifts are also there. Again, if we go to the gifts, it is likely an in kind gift. Yep, it was a $0 gift. So that was likely an in kind gift. And there we are. Okay. All right. So let’s go back to utilities and merge duplicates. Now, this time, we’re going to do the simple combine, the simple combine is just that it gives you little information. But as we told you fields that are populated or not are going to change depending on how we’re doing things. So let’s go in. And we’re going to do the we’re going to do, we’re going to take off the zip and the first name. And we’re going to do the simple combine. Right, so we’re going to run our report, I’m just keeping last name and address together, we’re going to click on Show Report. And we see we have three here, the one I’m going to work on is munchie zone. So if you take a look at munchie zone, we have the name, we have the address, we have Mr. Sal Smith here and the optional line, but it’s not here, we have a second address line. And we have the plus four zip code. So I’m going to do a simple combine. And again, we see very little information. But remember, fields will be updated depending on how we move the records. So let’s assume that I want to keep this record. Okay, if I keep this record, then what should happen is the optional line will populate in this blank field for the optional line, the address to line will not be touched. Because this is a blank field and blank fields do not overwrite data that are in the fields already. Okay, it’s going to combine the gifts. And in this case, it’s going to update the zip code to the plus four zip code. Okay, so what we need to do is select the radio button pointing to the record we want to keep. And now we click one combine. It asks if I want to combine the records. Yes, I do. And now they are combined. So if I go back to record 301. Now, we can take a look at what happened to good ol munchie zone. All right. Now, it did not take that plus four zip code. So I was mistaken there, it did not take the plus four zip code. So I could at this point, verify the address to get the plus four. But it did not overwrite that however, it did put the optional line in which is now contact on this organization record. It did not upset the address to line. Okay, but it did what we needed to do. All right.

So once you do your duplicates, you’re just going to keep going back to your list and moving the duplicates together. Let me just show you this once again. So you can see you’re going to do these in pairs. And as you do them in pairs, you’ll be able to merge the records, and then come back and go back to your list and do them. So they are done in pairs if you have multiple. And I’d like to show you that. So let’s just click the back button here. Let’s say I do first name. Let’s do this. For example, we’re going to do last name and first name. And I just want you to see, for example, Roger sandstone. So you’ll see Roger sandstone is matching up with three additional records. So what I’m going to do is take each one and compare it. So I might keep this one and move this one into this one, then these two will be compared together than these two. So that’s how it works. You’re just going to keep doing those in pairs. Now remember, when you’re doing your duplicates, you may need to do some research to see if in fact these folks are duplicates or not. A lot of people have similar names. You’ll see people that have addresses in different cities. They might be sunburned, you know sunbirds that go snowbirds, I should say, they might, you know, have a different address. So you really need to do your research to make sure that you’re not combining records, or you are combining records as you should be. Alright. So remember, always create a backup before cleaning up duplicates, as well as any other major changes to your system, like importing global update screen designer, even creating calculated fields. You want to periodically check for duplicates, especially before you do any mailings. After you do imports and after you download, download, online forms, all of those use matching criteria, especially imports in online forums. And those are the times when duplicates can happen. So make sure you’re doing and checking for duplicates periodically. I want to thank you so much for attending this. I hope that you enjoyed this webinar and that it will be helpful and keeping your database nice and clean and free of duplicates. Have a great Day. Bye Bye