AI Web Scraper

Most scraping breaks on the same thing: the site changes its markup and your selectors stop matching. AI Web Scraper skips selectors entirely. You give it a URL and a sentence describing what you want, and it reads the page the way a person would, then writes structured data back into your table.

Use it for the sources no dedicated scraper covers — a niche directory, a marketplace category, a listings page, a competitor’s customer page.

What it does

• Takes a start URL and an instruction written in plain English
• Crawls up to your page limit from that start URL
• Returns structured data — one field per thing you asked for
• Costs 0.4 credits per page scraped, so the default run is 2 credits per row

Like every enrichment, it is an Action column: it reads an existing column and writes its own, supports dynamic variables, and can fire on Auto Run.

Configuration

Required

Optional

Templates

Templates work the same way they do in AI prompts. Instead of rewriting an instruction each time you set up a column:

• Pick a public template — a ready-made instruction for a common source, loaded into the field and editable before you run it
• Save your own — once an instruction gives you the output shape you want, save it and reuse it on the next table

Table-level templates are a separate thing — those duplicate an entire workflow. See Templates.

How to use it

1. Click Add Action and search “AI Web Scraper”
2. Set Start URL — map it to a column with {{ }} if each row has a different site
3. Write the instruction, or load a template
4. Set Max pages if 5 is the wrong number for the source
5. Test on the first row with the play button, and check the fields that come back
6. Run the whole column from the ▶️ in the column header

Writing a good instruction

The instruction is the whole configuration, so be specific about the output, not the page.

• Name the fields. “Return company name, pricing tier, monthly price, and seat limit” beats “get their pricing”.
• Say what to skip. “Ignore blog posts and the careers page” keeps the page budget on pages that matter.
• Ask for one shape per column. If you want listings and contact details, that is usually two columns, not one instruction.
• Say what an empty value looks like. “Return empty if the page has no pricing” stops the model inventing a plausible number.
• Keep it short. A paragraph of context costs you nothing in credits, but a vague sentence buried in it costs you a re-run.

Common use cases

• Pull listings out of a niche directory or marketplace no built-in scraper covers
• Read a prospect’s pricing page into tiers and prices for qualification
• Extract the logos or names from a competitor’s customers page
• Monitor a jobs or announcements page and write the result into a signals column
• Turn a conference or event site’s speaker list into rows

Best practices

• Start with max pages = 1 and confirm the output shape before raising it — every page is billed
• Point it at the deepest URL that already holds the data, not the homepage; crawling to find the page spends your budget
• Filter the table down first, then run the column — this is a shortlist action, not a whole-table one
• Pair with Auto Run triggered off a column, so it only fires on rows that have a URL
• Use a built-in enrichment when one exists — it is cheaper and more consistent than scraping the same fact
• Reach for Website Contact Scraper instead when all you need is contact details; it is half the price per page
• Re-test after a source site redesigns; the instruction survives markup changes better than selectors, but not a page that moved

Where to next