Match Fields and Dedupe

Outcome: duplicates identified with a chosen survivor per group, a field mapping between your table and the CRM, and nothing merged automatically that a human has not confirmed.

Surface: Sync GTM app
Level: Intermediate
Uses: Formula columns · CRM integration
Credits: 0
Prerequisite: Lesson 02's imported slice

Normalize before you compare

Duplicate detection is only as good as the normalization underneath it.

Field	Normalization
Domain	Lowercase, strip protocol, `www.`, path and query
Email	Lowercase, trim
LinkedIn URL	Lowercase, canonical `/in/` form, strip query and trailing slash
Company name	Trim, fix casing, strip legal suffix into its own column
Person name	Trim, fix casing, split first and last

Do all of this into new columns. Never overwrite the CRM’s value — you may need to write it back unchanged, and you certainly need it to compare against.

The match key

Record type	Key	Fallback
Account	Normalized root domain	Company LinkedIn URL
Contact	Normalized email	LinkedIn profile URL
Contact, no email	LinkedIn profile URL	Name + account key

Never dedupe accounts on company name. “Acme”, “Acme Inc.”, “ACME Corporation” and “Acme, Inc” are four strings and one company — and worse, two genuinely different companies can share a name. A false merge is far more damaging than a duplicate: it is invisible in the table, it corrupts every number downstream, and it cannot be reliably undone.

Finding the duplicates

Group by the normalized key

Any group of more than one is a duplicate set.

Count and inspect

Read twenty groups by hand. You will find patterns — a bad import, a form that created contacts without checking, a merge that half-completed.

Classify the cause

Duplicates from one cause can often be fixed in bulk. Duplicates from many causes need review.

Flag near-matches separately

Same company name, different domain. Same person name, different email. These go to a review queue, never to an automatic merge.

Choosing a survivor

For each duplicate group, one record survives and the others are merged into it. Choose by rule, not by whichever appears first.

Rule	Priority
Has an open opportunity	Highest — never merge away an active deal
Most recent activity	Strong
Most complete — fewest empty fields	Strong
Oldest created date	Weak, but a reasonable tiebreak
Has a named owner	Prefer over unowned

Then merge field by field: take the most recent value for volatile fields, the verified value for contact data, and never overwrite anything a human wrote — notes, disqualification reasons, custom text.

Field mapping

Write the mapping down before any repair work. For each field:


crm_field          the destination
table_column       the source
source_of_truth    crm | syncgtm | verified-wins
on_conflict        newest | verified | never-overwrite

That table is what lesson 06 executes. Making the decisions now, in the calm, is what prevents an argument later about why a rep’s note disappeared.

Do this now

Build the normalization columns

Into new columns, never overwriting.

Set the match key per record type

Group and count duplicates

Record the number — it is one of the findings from lesson 01.

Read twenty duplicate groups

Classify the causes.

Apply the survivor rules

One survivor per group, chosen by rule.

Route near-matches to review

Never auto-merge them.

Write the field mapping table

Every field you intend to touch.

Check your work

Normalization went into new columns
The match key is a domain, email or profile URL — never a name
Every duplicate group has a chosen survivor
Near-matches are queued for human review
The field mapping table exists, with a source of truth per field

Where this breaks

Merging an account that has an open opportunity into another record can move or orphan the deal, depending on your CRM. That is a revenue-affecting mistake, and it is entirely avoidable: check for open opportunities before any merge and exclude those groups from automatic handling. If both records in a group have opportunities, a human decides.

Further automation

Fix duplicates at the source as well as in the data. Most duplicate creation traces to two or three causes — an unchecked form, an import without dedupe, a sync creating rather than matching. Cleaning the data without fixing the cause means doing this again next quarter.

Next lesson

04 — Fill the gaps, completing empty fields without touching populated ones.

Reference for this lesson: CRM integrations, Import, Tables, GTM Engineering — dedupe and hygiene.