Bulk merge duplicate people, companies

Table of Contents

Overview

Find duplicate people, companies, deals in flexible ways. Then, generate a CSV report of the duplicates.

Simulate the deduplication process by running it in "preview" mode before actually merging duplicates in the CRM to make sure your master selection rules work as expected.

Run deduplication in bulk, then automate the process to keep your database free from duplicates at all times.

Quick Start using Templates

Open the Merge Duplicates module and pick a record type, for example, contact or company.

For a quick start, explore the default templates. There might be a template that already solves exactly what you need. Alternatively, use an existing template as a starting point and adjust it to your requirements.

Step 1: Find Duplicates

In order to find duplicates, you need to define how to match records. For example, to find duplicate people you may use "First Name", "Last Name", and "Email Domain" fields. In that example, people with the same first name AND same last name AND email domain will show as possible duplicates.

Pick the relevant fields to match records by, and optionally define how to match the values of those fields:

  • Comparison Rule: Pick between "Exact Match" and "Similar Match", also known as fuzzy matching to take into account typos.

  • Ignored: Pick parts of the value to ignore. For example, ignore symbols and whitespace when comparing phone numbers, or ignore HTTP, www, subdomain, or top-level domain (.com vs co.uk) when comparing websites or email domains.

  • Match Parts: Pick whether to compare the entire value, any two words, or just the first 5 letters, or X last characters, etc.

You can add fields by clicking the + button, or remove fields by clicking the X button.

When you click "Find", Insycle will look for records whose values match ALL the fields.

Optional advanced settings:

  • Filter: segment the data prior to analyzing for duplicates. For example, only look for duplicates in email domains that are not free email providers (Gmail, Hotmail, Yahoo, etc.), or records owned by Jane, or people whose name starts with a

  • Custom terms and text to ignore: specify additional terms (standalone words) or plain text to ignore when comparing values. For example, inc and incorporated, http://, https://, www.

Step 2: Review Duplicates

Here you can review the duplicate records that have been identified using the fields and matching rules you selected in Step 1.

When two or more records represent the same entity (person, company, or other), we refer to them as a Duplicate Group. For example, if you had 4 records for the same person, it would count as 1 Duplicate Group with 4 records, out of which 3 are duplicates.

In this view, each row represents a Duplicate Group. When you click a row, it expands and shows the records that belong to that group.

You can add more columns to the view using the ⚙ button on the right, in order to get more context and analyze the records.

To create a CSV report of all duplicates, see the steps below.

Step 3: Merge Mode

You can merge duplicates in bulk using rules, or manually one by one.

When merging in bulk, Insycle limits the duplicate group size to 5 and skips groups that contain more records. For example, Insycle will not merge a group of 7 records that represent the same person. You can adjust the default value of 5 as needed from 2 and up to 100 records per Duplicate Group.

Step 4: Master Selection

The goal is to pick one record from each Duplicate Group as the master, and the remaining records in that group will get merged into it. For example, if you had 4 records that represent the same person, it would count as 1 Duplicate Group with 4 records, out of which 3 are duplicates that will be merged into 1 master record.

When merging in bulk, the master record from each Duplicate Group is determined using rules via an elimination process. For each Duplicate Group, Insycle eliminates records that do not meet the rule criteria. Rules are read in order, from top to bottom.

For example, let's say that we have 4 records that represent the same person Britany Calvert, that is, 1 Duplicate Group that consists of 4 records. We haven't sent any emails to Britany yet, so when Insycle processes the first 3 rules: email clicked, email opened, and email bounced, Insycle cannot eliminate any record because they all have the same value of zero. In the next rule about contact owner, records 1000, 1001, and 1003 are eliminated since no contact owner exists for those records. Now there is only one record remaining 1002 and therefore that's the master record.

You can add or remove rules using standard and custom fields. And since the order of the rules is important, you can also reorder the rules as needed using the arrow buttons.

Record ID

Name

Email

Contact Owner

1000

Britany Calvert

britany@acme.com

1001

Britany Calvert

bcalvert@acme.com

1002

Britany Calvert

britany.alvert@acme.co.uk

Jane Adams

1003

Britany Calvert

britany.alvert@gmail.com

Review

Now you can generate a CSV report, and run a simulation of the deduplication process - before actually merging duplicates in the CRM - to make sure your master selection rules work as expected.

Click the Review button which will open the following dialog. You can choose between "Preview" mode which simulates the deduplication process for analysis purposes without modifying any data in the CRM, and "Update" mode which updates the data in the CRM.

Click the "Next" button to add email recipients that will receive the CSV duplicates report, and customize the message subject and body.

Finally, click the "Next" button to choose between running the process automatically on a recurring basis, or running it one-time right now.

Insycle will run the deduplication process according to your selection and you'd receive an email with a CSV report of the duplicates.

Once you're ready to bulk merge duplicates in the CRM, click the "Review" button again and pick "Update" mode. Insycle will merge duplicates in the CRM directly.

Tip: every field that is specified in Step 4 Master Selection is also included in the CSV report, and so you may want to specify additional fields to get more context about the records in the CSV report. For example, add "Industry exists" as the last rule to have the "Industry" field included in the CSV report.

Audit Trail and History

The Activity Tracker lets you review all changes made through Insycle. At any time you can download a CSV report of the operation and records affected.

Automation

Maintaining your CRM data plays a critical role in the effectiveness of any team that uses that data. Insycle automates CRM data maintenance so that you can focus on optimizing processes, not running complicated data maintenance tasks by-hand.

Save the configuration you just created into a template, so that you can reuse it in the future, collaborate with your team on the settings, and ultimately automate tasks.

You can choose to automate a template to run on a recurring basis based on a schedule, or integrate it into a workflow automation builder in your CRM.

Data Health Assessment

The goal of the Data Health Assessment is to surface data quality issues that negatively impact your marketing, sales, and support efforts, and to guide you through the process of fixing them.

Review duplicates that have been identified automatically, and then add your own custom configurations to the report, in order to track duplicate count and progress and trends over time.

Manual Merge

In some cases, you may want to merge duplicates manually one by one. To do so, pick "Manual Mode" in Step 3. Then, in Step 2 choose the relevant records to merge, and at the bottom of the page pick the master record, and optionally specific field values to override.

Customizations

For situations where you need more granular customization for picking duplicate records to include - or exclude - from the deduplication process, or for picking the master record, and there are no common rules you can apply for all or some of the records, you can customize bulk deduplication using exclusions and pre-defined masters.

Separately, it is also possible to customize the merge logic behavior. For example, to instruct Insycle to keep values from the master record instead of the most recently updated value, or copy values from one field to another as part of the merge, or any other. Customized merge logic requires an Enterprise plan. Please reach out via chat to discuss your specific requirements.

Additional Resources

CRM-specific Deduplication Help Articles

Did this answer your question?