Duplicates Part 2 – Identifying Duplicates in Your Database

This is the second post in a three-part series about duplicate management.

Just like the weeds in Devendra’s garden, duplicates happen! Even if we have preventative measures in place, your Altru database is being touched by your whole organization and even your patrons online! Being armed with effective ways to manage those duplicates will help keep your database clean and your constituent garden weed-free!

There are two primary components to managing duplicates in your Altru database:

1. 
   
2. Identifying Duplicates (which we will discuss in this blog post)
   
3. Merging Duplicates (which we will talk about next week!)
   

Altru identifies duplicates through the Duplicate Search Processes. There are two duplicate search processes that you can take advantage of:

• 
  
• Full Duplicate Search – this process scans your entire database for potential duplicates.
  
• Incremental Duplicate Search – this process scans all records that have been added or changed since the last search process. This option can save on processing time if you have lots of constituent records!
  

Within the Search Processes, you have three search settings to find help duplicates:

1. 
   
2. The Fast Process: This process searches on most criteria to return the highest probable matches in your database. This process also takes the least time if you’re looking for a quick scan of your database. Because the Fast Process is so specific, it will generate fewer results, so you may need to follow up with a Detailed Search to find more potential duplicates.
   
3. The Detailed Process: This process searches on the fewest criteria to return the most possible matches. This will take longer and you’ll have more to sort through, but it is a great deep scan of your database!
   
4. Custom Settings: With the Custom Settings option, you can choose what elements of the constituents you would like to match and pre-filter on. My favorite part of the Custom Settings is the option is to include a selection.
   

For example, if you have TONS of duplicates to sort through, you can use Custom Settings with a selection to filter on only organizations, or only constituents with a membership. This will help minimize your list of duplicates and help you get through your merge processes faster.

Once the search is complete, the Duplicate Constituent Report can help you review your potential matches.
I also recommend using this report to see if any relationships or households need to be created. For example, Cristina and Chris appear here in our results—they have similar names, but aren’t the same person. You can also click the hyperlinked name of the constituent to go to their record.

If this is your first time merging duplicate in your database, here’s what I recommend:

1. 
   
2. Run the Full Duplicate Search Process with the Fast Process and merge. This gets the highest probable matches merged and out of your way!
   
3. Run the Full Duplicate Search Process with the Detailed Process and merge. This will give you more to review, but this will help to really get your database squeaky clean!
   

Once you’ve done this initial merge, you can then put your Duplicate Search processes on a job schedule. This means they will run automatically and you won’t have to worry about manually running the process before you merge—a great timesaver!

Even after exploring these reports, you may need to set up some audit queries to help identify duplicates that won’t be picked up in the duplicate search process. For example, if a constituent is missing a key piece of information (such as an address or first name), they won’t likely have a 70% or above match percentage to a more complete record and therefore won’t show in our duplicate search process. We’ve actually pre-built a few queries in your Altru databases to help with this:

• 
  
• Constituents with no primary address – helps to find constituents without an address. These may get missed in the duplicate search process.
  
• Individuals with No First Name – pulls all individual constituents that do not have a first name.
  
• Constituents added between date range – this query is designed to help audit data that is being entered. You can adjust the date range to monitor your data entry.
  
• Spouse Relationship Type but Is Spouse not set – this query is designed to help find constituents that may have a spouse relationship, but the “Is spouse” box wasn’t checked when adding the record.
  

For more information about running the Duplicate Search Process and identifying duplicates, see our Duplicate Management Curriculum here!