Scraping a Single Page
Outcome: a working scrape column that builds its own URL per row, returns a named fact, and has been verified by hand against five live sites.
- Surface
- Sync GTM app and MCP server
- Level
- Beginner
- Uses
- Web scraper column · scrape_pricing_plans
- Credits
- ~5 for 10 rows
- Prerequisite
- Lesson 02
Build the URL, do not type it
The scrape target changes per row, so it comes from a formula column, not a fixed value.
scrape_url = "https://" + domain + "/pricing"Three reasons to make this its own column rather than inlining it:
- It is free, and you can read it to see exactly what was requested.
- A failing row shows you the malformed URL immediately.
- Changing the path across the whole table is one edit.
Handle the variants your list actually contains: /pricing, /plans, /pricing-plans. Either try the most common one and fall back, or use the dedicated scrape pricing plans action, which handles the variation for you.
High-signal pages
| Page | The fact worth extracting |
|---|---|
/pricing | Is a price published? Lowest tier? Is there a free plan? Is it per-seat? |
/integrations | Do they integrate with your category? With you? With your competitor? |
/customers | Which logos — their real ICP, in their own words |
/careers | Which roles, which locations, how many |
/about | Founded date, team size, offices |
/security or /trust | SOC 2, ISO — a strong enterprise-readiness signal |
/changelog or /blog | Whether the product is actively shipping |
The best of these are the ones your competitors’ lists do not have. Anyone can buy headcount; nobody else on your prospect’s inbox has read their integrations page.
Extract, do not dump
A scrape returns a page. What you want is a value. Two patterns:
Pattern match, when the shape is predictable
A formula or regex over the scraped text: a currency figure, a “Free” string, a known vendor name. Free, fast, deterministic.
AI over the scraped text, when it is not
An AI column reading only the scraped output — not fetching anything itself. This grounds the answer in real content instead of the model’s memory, and it is much more reliable than an unanchored research call.
The second pattern is the workhorse of this course: one fetch, one constrained answer. Lesson 06 covers the prompt.
Keep the raw scrape in its own column even after extracting. When an answer looks wrong, the raw text is the only way to tell whether the scrape or the extraction was at fault — and re-fetching to find out costs credits.
Verify against the live site
The only real test. Take five rows, open each URL in a browser, and compare the extracted value to what you see.
Count three things:
- Fetch success — did you get page content at all?
- Extraction accuracy — is the value right on the rows that fetched?
- Absence handling — where the fact genuinely is not on the page, did you get a clean “not found” or an invented answer?
The third matters most. A pricing page with no price should yield not published, which is itself useful information.
Do this now
Add the URL formula column
Domain plus path. Read ten of them and confirm they look like URLs you could paste.
Scrape five rows
Read the raw output for each.
Add a status and length check
Flag anything short, and anything containing 404 markers.
Extract the value
Pattern match if the shape is predictable, AI over the scraped text if not.
Open all five sites yourself
Compare. Fix the extraction, not the data, where they disagree.
Add the fallback
For failed fetches: try a second path variant, or route to a research agent in lesson 05.
Widen
Only qualified rows, and check the failure rate after the run.
Check your work
- The URL comes from a formula column you can read
- Raw scrape output is kept alongside the extracted value
- Five rows verified by hand against the live site
- Pages that genuinely lack the fact return a clean absence value
- Failure rate after the full run is recorded
Where this breaks
Assuming a missing page means a missing fact is the error that produces confidently wrong segmentation. No /pricing page might mean enterprise sales — or it might mean the pricing lives at /plans. One is a qualification signal, the other is a bug in your URL formula, and in the table they look identical. Try at least two path variants before recording an absence.
Further automation
Scraped facts have different shelf lives: pricing and integrations change quarterly, a changelog weekly. Set the refresh per column, and let a change in the scraped value become its own signal — a company that just published pricing, or added your competitor as an integration, is worth a message that week.
Next lesson
04 — Pulling emails and phones from a site, the scrape that recovers contact data when enrichment finds nothing.
Reference for this lesson: Web scrapers, Scrape pricing plans, Scrape domain details, AI agents.