Dedupe, Hygiene and the Source of Truth
Outcome: a match key, a written merge rule, a refresh cadence per field, and a suppression list — the four things that let a dataset be re-run weekly without corrupting itself.
- Surface
- App
- Level
- Beginner
- Uses
- Formula columns · import dedupe · CRM integration
- Credits
- 0
- Prerequisite
- Lesson 02's grain decision
Why this is a lesson and not a footnote
A one-off list does not need hygiene. A pipeline does, because it re-runs — and every re-run is an opportunity to create a second copy of a record, re-bill for data you already bought, or email someone for the second time in a week.
Four decisions cover it. Make them before the second run, not after.
1. The match key
Pick one identifier per grain and normalize it. Everything else is a display field.
| Grain | Match key | Normalization |
|---|---|---|
| Company | Root domain | Lowercase, strip www., strip protocol and path, keep the registrable domain |
| Person | LinkedIn profile URL | Lowercase, strip query string and trailing slash, use the canonical /in/ form |
| Person, fallback | Verified work email | Lowercase, trim |
| Event | Source ID plus company key | Whatever the source provides, prefixed by type |
Never match on company name. “Acme”, “Acme Inc.”, “Acme, Inc” and “ACME Corporation” are four rows and one company, and no amount of string cleaning fixes the general case. Name matching is the single most common cause of duplicate accounts in GTM datasets.
Two domain edge cases worth deciding up front:
- Subsidiaries and parents.
eu.acme.comandacme.commay or may not be one account. Decide by how you sell, and write the rule down. - Shared-host domains. Freelancers and small businesses on a platform domain will collapse into one row if you match on domain alone. For SMB lists, match on domain plus business name.
2. The merge rule
When a re-run produces a row that matches an existing one, something has to win. Decide per field type, not per field.
Newest wins — for volatile facts
Headcount, funding, job title, tech stack. The new value is more likely to be true.
Verified beats unverified — for contact data
A verified email never gets overwritten by an unverified one, regardless of age. Status ranks above recency here.
Never overwrite human input
Anything a person typed or corrected — an account owner, a note, a manual disqualification — is immune to automated writes. Give these fields a naming convention so a merge rule can recognize them.
Append, never replace, for events
Signals accumulate. A new funding round does not overwrite the previous one; it becomes a second event with its own date.
3. Refresh cadence
Re-enriching everything on every run is the most common way to burn credits for no gain. Match cadence to how fast the field actually changes.
| Field | Cadence | Why |
|---|---|---|
| Signals | Weekly | The whole point; windows are short |
| Headcount, job listings, hiring | Monthly | Moves quarter to quarter |
| Funding, tech stack, revenue | Quarterly | Slow, and expensive to re-check |
| Company name, domain, country | Never, unless flagged | Effectively static |
| Verified email | On bounce, or every 6 months | Verification decays; addresses mostly do not |
| Person’s role | Quarterly, or on a job-change signal | The signal is cheaper than the re-enrich |
The last row generalizes: detecting change is usually cheaper than re-fetching the record. Check whether a person moved, and only re-enrich the ones who did. That single pattern is most of the Job Changes course.
4. Suppression
One list, checked before every send, containing:
- Current customers, unless the play is expansion
- Open opportunities owned by someone else
- Anyone contacted in the last 30 days by any campaign
- Unsubscribes and hard bounces — permanent, non-negotiable
- Competitors, partners, investors, and your own domain
- Manually disqualified accounts, with the reason recorded
Suppression is checked at the Outreach boundary, not at Find. Suppressing early hides accounts from your own reporting and makes market-size numbers wrong.
The source of truth
Two systems will disagree. Decide in advance which one wins for which field, and write it as a table.
| Field | Source of truth | Reason |
|---|---|---|
| Account owner, stage, deal value | CRM | The business runs on it |
| Headcount, tech stack, funding, signals | Sync GTM | It refreshes; the CRM does not |
| Email address | Whichever is verified more recently | Verification decides, not system of origin |
| Notes, disqualifications | CRM | Human-entered |
Without this table, the enrichment pipeline and the CRM overwrite each other on a schedule and both become untrustworthy. CRM integrations covers the write mechanics; CRM Enrichment is the full build.
Do this now
- Write your match key and its normalization rule as one sentence.
- Add a normalized key column to your table and check for collisions on the rows you already have.
- Write your merge rules — four lines is enough, one per field type.
- Tag every column with a refresh cadence.
- Build the suppression list, even if it is only unsubscribes and current customers today.
- Fill in the source-of-truth table for every field that exists in both Sync GTM and your CRM.
Check your work
- Your match key is a domain or a profile URL, not a name
- You can say what happens when a re-run hits an existing row, per field type
- No column is set to refresh on every run unless it is a signal
- Suppression is checked at the Outreach boundary
Where this breaks
The expensive failure is re-enriching on every scheduled run because no cadence was set. A 5,000-row table with four enrichment columns re-run weekly is four times the annual spend of the same table re-run monthly, for data that changes quarterly at best. Set cadences before you schedule anything, and check the credit consumption on the second run specifically — it is the run where this mistake becomes visible.
Further automation
The mature version writes hygiene back rather than just enforcing it locally: dedupe on the way into the CRM, flag conflicting records for review, and let the pipeline own the fields it is the source of truth for. That is a build, not a setting — CRM Enrichment walks through it.
Next lesson
08 — Unit economics: cost per contactable lead, which turns every number in this course into one you can defend.
Reference for this lesson: Import, CRM integrations, Workspace, Verify email.