Duplicates Part 3 – Merging Duplicates

This is the third and final post in a three-part series about duplicate management.

If you look at the images above, you’ll notice they are actually the same person. Go ahead – look closely.

I’ll wait.

Every database has duplicates. It doesn’t matter what system it is or what company it comes from. It’s just one of those things that will always be an issue because human beings are somehow involved. If we accept that duplication happens, we’ll have a much easier time of dealing with it.

Identifying duplicates

Occasionally when searching for a single record, you’ll trip over something like this – either a definite or suspected pair of duplicates. In the absence of a concerted effort to locate and deal with duplicates this is the most common way of discovering them.

When duplicates are found in this manner, your organization will need a policy to know what to do next. Do you merge them since you found them? Notify someone else? Do nothing?

Merging Two Duplicates

Well great. We’ve got duplicates. Now what? There are two ways to merge duplicates in Altru:

• 
  
• Merge individually identified pairs of records.
  
• Bulk merge records based on a system-wide search.
  

In this post we’ll tackle both. First, how do we deal with a single pair of duplicates we trip over when going through our day-to-day tasks?

 

Let's have a look at the duplicate merge utility. Navigate to Constituents > Duplicates. There are two steps to merge a record:

1. Check your merge configuration settings. These are the rules Altru uses to decide what information to merge and what to discard. The default merge configuration is used by most organizations with very few, if any, changes.


Click Edit to review or change this configuration.

Each checkbox represents a different type of data that can be stored on a constituent record. If present, the checkbox, and any further settings inside the selected item define the rules Altru will follow when merging that particular slice of data.

The screenshot shows how addresses will be handled. By default the Target record’s address data will be kept as primary, and the Source data will be stored as additional addresses. In the case of an exact match on address, the Source address will be discarded. The rest of the settings follow a similar pattern. You can tweak these, or create a whole new set of values, saved under a different name.
Once you are satisfied that the settings accurately reflect how you want Altru to behave, click Save.

2. Merge two records. Click Merge two duplicates and fill in the form. Remember that Source is the yummy snack. Target is the record that is hungry and wants to eat a yummy snack.
Once you set your values and press Merge, you’ll be taken to the remaining record. Make sure to peruse the tabs, particularly contact info, to look for any remaining duplicated items like addresses, phone numbers and email addresses. Those will need to be manually cleaned up.

Duplicates are always part of a system like Altru, but a tool like this can help you quickly and easily get rid of them as they are discovered as part of normal day-to-day activities.

We’re infested – Mass-merging duplicates

Using the Full or Incremental Duplicate Search function, we can get Altru to do some serious self-reflection and determine on its own the state of its duplicate records. We discussed that in detail in the previous post.

Once the process has run, the next step is to use the results to merge many pairs of duplicates together at the same time. This is a much less time-consuming option that explicitly merging two records at a time.

From Duplicates, click Constituent Merge. We can create any number of processes here, each with its own rules, and levels of sensitivity in handling potential duplicates. In this example we will create and run a very restrictive 100% match process. It will only look at the most confident matches it found, and ignore all others for a later time. This is an incremental approach to mass-merging duplicates, and will help you build confidence in using the tool.

1. 
   
2. Click Add.
   
3. Name the process.
   
4. Select a Merge configuration – we’ll use the default configuration we looked at earlier.
   
5. Duplicate record source – this is how Altru knows what record pairs are to be matched. The sensitivity of the Full or Incremental Duplicate Search report the last time it was run determines that group of records.
   
6. Target constituent criteria – in each pair to merge there must be a Source and a Target. The rules here determine how Altru decides which of the pair is which. Generally speaking most orgs will want to keep the oldest record – the one that was created first, as the duplicate exists because a proper search was not performed.
   
7. Excluded constituents – we can build a query to explicitly exclude records we do not want to merge. Later on we’ll also see how to manually exclude records before we run the process.
   
8. Minimum match % - let’s set this way up at 100%. In subsequent runs of the process you can lower the value to include more records.
   
9. Only merge individuals – this will tell Altru to ignore any potential Organization record matches.
   
10. Omit individual/organization matches – this will tell Altru to ignore an Individual record that is interpreted as a duplicate with an Organization record.
    
11. Source constituent – we can either delete the source or mark it inactive. The purpose for merging duplicates is to clean up the database, removing the extra records we don’t need. Unless you have a really good reason to keep records as inactive, we recommend deleting them.
    
12. Click Save.
    

Now that our new process exists, let’s click on the name of the process to get more information before we run it.
Click on the Preview Merge tab.

We now have a very granular ability to review the potential matches that will be merged, and exclude any pairings from the merge process if we have any doubts about whether they are really matches. Click the Do not merge button to exclude a pair of records from the next run of the process. Click Delete merge to remove the match from the process entirely. Note that the pair will be re-added the next time you run the full duplicate search process.

Once we have reviewed our records, we can start the process!

Click Start process, review the settings, and click Start when ready.
Our process has completed!
Just one record (for each duplicate match) remains…
…how you get it out of your bed, is up to you.