Duplicate data in HubSpot poses serious problems for any company.

Duplicate HubSpot records inhibit your marketing team from effectively segmenting and personalizing your communications. Sales teams step on each other's toes and lack vital context in conversations. Support teams miss important information, and analysis and reporting are skewed.

Insycle helps you to merge duplicate contacts, companies, and deals—flexibly and powerfully.

Table of Contents

Quick Summary

With Insycle, you can identify and merge duplicate contacts, companies, deals in flexible and powerful ways.

Insycle can identify duplicates using any field in your database, with flexible matching rules. Once Insycle identifies the duplicate records, you then set rules for determining the master record that other duplicates will be merged into—such as the first record created, record with the most email opens, or any other field that would be relevant for determining the right master record. You can also set merging logic on a field-by-field basis.

With Insycle, you can run deduplication processes in bulk, then automate the process using templates to keep your database free from duplicates at all times.

Step 1: Find Duplicates

Insycle deduplication takes place in the Merge Duplicates module. Navigate to there and from the top of the screen, choose whether you will be deduplicating contacts, companies, deals, or custom objects.

For a quick start, explore the default templates. There might be a template that already solves exactly what you need. Alternatively, you can use an existing template as a starting point and adjust it to your requirements.

Deduplication Templates

In order to find duplicates, you need to define how to match records.

For example, to find duplicate people you may use "First Name", "Last Name", and "Email Domain" fields. In that example, people with the same first name AND same last name AND email domain will show as possible duplicates.

Matching rules

Insycle allows for deep control in defining how to match the values of those fields.

You can add fields by clicking the "+" button, or remove fields by clicking the X button.

When you click "Find", Insycle will identify duplicate records based on the matching filter that you have set in this step.

Step 2: Review Duplicates

In this Step, Insycle allows you to preview the identified duplicates and double-check to make sure that your filter is producing accurate results. That way, you can avoid merging any records that are not actually duplicated.

Once you click the "Find" button in Step 1, Insycle will generate a list of the identified duplicates for you to review.

When two or more records represent the same entity (person, company, or other), we refer to them as a Duplicate Group. For example, if you had 4 records for the same person, it would count as 1 Duplicate Group with 4 records, out of which 3 are duplicates.

In this view, each row represents a Duplicate Group. When you click a row, it expands and shows the records that belong to that group.

You can add more columns to the view using the ⚙ button on the right, in order to get more context and analyze the records.

Review duplicates

Clicking the checkbox on a record expands your view and allows you to look at all identified duplicates within that Duplicate Group.

Duplicate group

Here, you can dig deep into your identified duplicate groups, down into the individual records, to check for accuracy and get a better idea of the best way to determine the Master Record that other records will be merged into, which we will do in Step 4.

Additionally, this is where you can select records that you would like to merge manually, which can be chosen in Step 3.

Step 3: Choose Operation

In Step 3, you can merge duplicates in bulk using rules, or manually one by one.

When merging in bulk, Insycle limits the duplicate group size to 5 and skips groups that contain more records. For example, Insycle will not merge a group of 7 records that represent the same person. You can adjust the default value of 5 as needed from 2 and up to 100 records per Duplicate Group.

This is a safety precaution to make sure that if you use a duplicate matching filter that is too broad in Step #1, that you do not accidentally merge many non-duplicate records together.

Maximum records in duplicate group bulk merge

If you select Manual Mode, you'll have complete control over which records are merged together, by selecting them from the Record Viewer in Step 2.

duplicate group selections

In Manual Mode, you can also individually select the Master Record that other records will be merged into on a case-by-case basis.

master record

In Bulk Mode, you set rules for determining the master record automatically in Step 4.

Step 4: Master Selection

Insycle offers extremely flexible controls from determining the master record that all records in a duplicate group will merge into.

You can make your master selection on the record level, such as merging all duplicates in a group into the earliest created record.

Or, you can customize your master selection down to the field level. For example, maybe you want the "Email" field to come from the last modified record, but you want First Name, Last Name, and Phone Number to come from the earliest created record.

Insycle gives you complete control over how data is merged into your master record using advanced rules.

Master Selection By Record

The goal is to pick one record from each Duplicate Group as the master, and the remaining records in that group will get merged into it. For example, if you had 4 records that represent the same person, it would count as 1 Duplicate Group with 4 records, out of which 3 are duplicates that will be merged into 1 master record.

When merging in bulk, the master record from each Duplicate Group is determined using rules via an elimination process. For each Duplicate Group, Insycle eliminates records that do not meet the rule criteria. Rules are read in order, from top to bottom. So if a Master Record meets the first criteria (Highest marketing emails clicked in the example below) the others on the list will not impact the selected Master Record.

bulk master selection

For example, let's say that we have 4 records that represent the same person — Britany Calvert. That is 1 Duplicate Group that consists of 4 records.

Here is the data that we have for this duplicate group:

Record ID

Name

Email

Contact Owner

1000

Britany Calvert

britany@acme.com

1001

Britany Calvert

bcalvert@acme.com

1002

Britany Calvert

britany.alvert@acme.co.uk

Jane Adams

1003

Britany Calvert

britany.alvert@gmail.com

We haven't sent any emails to Britany yet, so when Insycle processes the first 3 rules — emails clicked, email opened, and email bounced — Insycle cannot eliminate any record because they all have the same value of zero.

In the next rule about contact owner, records 1000, 1001, and 1003 are eliminated since no contact owner exists for those records. Now there is only one record remaining 1002 and therefore that's the master record.

You can add or remove rules using standard and custom fields. And since the order of the rules is important, you can also reorder the rules as needed using the arrow buttons.

Master Selection By Field

Insycle also gives you the ability to pick which data to retain in the master record on a field-by-field basis. Then, the master record will be decided based on the rules that you set but may pull values in fields from several different records from the duplicate group, based on the rules that you set in the Fields tab.

For instance, you might want the First Name and Last Name fields to be retained from the earliest created record in the duplicate group. But, you may want the email retained to come from the most recently updated record. With Insycle, you can. In this situation, the master record will keep all values for all additional fields, but the First Name, Last Name, and Email will be pulled from other records in the group based on your settings.

Or, you may want to retain the "owner" from the record that was created first.

Let's take a look:

By-field duplicate feature

You have multiple options for determining criteria:

  • From master record. The value for this field will always be retained from the master record that you set in the "Record" tab in this step.

  • From record where value. For example, retain "Employee #" field from the record where the value is highest.

  • From record based on other field. Retain a field based on criteria for a secondary field. In the example above, we retain the data in the Contact owner field from the record with the earliest create date.

In this example, we have set by-field master selection rules for 5 different fields; Annual Revenue, Employees, Contact owner, Lifecycle Stage, and My Custom Field.

In the case of Lifecycle Stage, there are two rules. Insycle will look at the first rule in the list first, and if it finds a record that matches the criteria, the second rule will be ignored. In the example above, if Insycle were to find a record in the duplicate group that had the "Lifecycle Stage" of "Customer," then the following rule looking for "SQL" would be ignored.

Using field-by-field master selection rules gives you unparalleled control over what data is retained in your master record during a bulk merge.

Step 5: Preview Or Update

Now, with our Filter and Master Record selection in place, we can run the Merge Duplicates module to update our selected records.

Click the Review button which will open up a pop, where you can choose between "Preview" mode which simulates the deduplication process for analysis purposes without modifying any data in HubSpot CRM, and "Update" mode which updates the data in the CRM.

merge duplicates preview and review

Initially, it is a good idea to run in Preview Mode and ensure that your deduplication template is working as you would expect. Once you have confirmed that it is, then you can update the live data in your database.

Click the "Next" button to go to the Notify screen, where you can select who will receive an email report of the Preview or Update. You can also add additional context here.

share report

Finally, click the "Next" button to choose whether you will schedule this deduplication process to run automatically on a set schedule, or one-time.

merge duplicates automation

Here, you can choose to schedule this operation to run hourly, daily, weekly, or monthly. When scheduled, the Module will automatically run on your desired schedule and email the reports after it has run. You can view all scheduled automations on the “Automations” page on your dashboard.

Tip: Every field that is specified in Step 4 Master Selection is also included in the CSV report, and so you may want to specify additional fields to get more context about the records in the CSV report. For example, add "Industry exists" as the last rule to have the "Industry" field included in the CSV report.

The CSV report generated provides a complete look at the duplicate group. In the "Result" column, the "Master (After)" for each duplicate group will show the data the final master record will contain, including master selection settings both for by-record and by-field configurations. You can see the entire duplicate group and the fields from them that were used in the master record.

csv report master

When a field says "(Default)," it means that HubSpot CRM will be using its default processes for dealing with the field—this is typically seen for blank fields, system IDs, and other specific situations.

Once you are satisfied with how the operation runs, you can save your settings as a template for future use by clicking on the "+" and "Save" buttons on the Template Menu.

templates

Preview Changes Before They Go Live

With Insycle, you can always preview the changes that you are making to your data before those changes are pushed to your live database. When you run modules in Insycle, you have the option of choosing between Preview Mode and Update Mode once you click the button.

Automation

You can set up ongoing data maintenance automation with Insycle on the module level, using Recipes, or integrating with Workflows. You can view all scheduled automations on the “Automations” page on your dashboard.

Audit Trail and History

The Activity Tracker lets you review all changes made through Insycle. At any time you can download a CSV report of the operation and records affected.

Data Health Assessment

The Data Health Assessment surfaces data quality issues that negatively impact your marketing, sales, and support efforts, and guides you through the process of fixing them. Here, you can keep an eye out for common duplicates or add your own custom configurations to the report, in order to track duplicate count and progress and trends over time.

Customizations

For situations where you need more granular customization for picking duplicate records to include - or exclude - from the deduplication process, or for picking the master record, and there are no common rules you can apply for all or some of the records, you can customize bulk deduplication using exclusions and pre-defined masters.

Separately, it is also possible to customize the merge logic behavior. For example, to instruct Insycle to copy values from field A to field B as part of the merge, or combine multiple fields into one. Customized merge logic requires an Enterprise plan. Please reach out via chat to discuss your specific requirements.

Related Blog Articles

Related Help Articles

Additional Resources



Did this answer your question?