Websites we scrape
Every site needs its own parser and its own upkeep, so we list what we actually run rather than claiming we handle everything. Below is what is live today, how we grade a new source, and what we turn down.
Running today
Both of these have been extracted end to end and published as free datasets. Open either one and you can judge our output without a conversation.
Gap.com
Full US apparel catalog: prices, markdowns, colour variants and size-level availability. Around 7,700 listings across 71 category paths. The parser adapts to Old Navy, Banana Republic and Athleta with little rework.
IKEA USA
Full US catalog: article numbers, prices, dimensions, materials and a category tree five levels deep. Around 8,800 products. Article numbers are global, so other IKEA markets are a cheap extension.
How we grade a new source
Price and reliability on a scraping project depend far more on how a site defends itself than on how many rows come out. We sort every target into one of three tiers before quoting anything.
| Tier | What it looks like | What it means for you |
|---|---|---|
| Straightforward | Server-rendered pages, structured data on the product page, no challenges on normal browsing, pagination that works | Fast to build, cheap to run, stable between redesigns. Full catalog in two working days. Gap and IKEA sit here. |
| Moderate | Commercial bot protection, capped pagination, prices or stock that depend on a location, data buried in framework state rather than markup | Workable and routine, but needs proxy rotation, a pinned location and category slicing. Longer build, higher running cost, more frequent parser upkeep. |
| Heavily defended | Aggressive challenges, address-level bans, signed API requests, marketplace structures where one product means many offers | Possible, but the economics change: residential addresses, headless browsers and ongoing rework. We take these case by case and quote them honestly rather than absorbing the risk quietly. |
Sources we are bringing online
We publish a site here once it has run cleanly for a while, not when we start work on it. Currently being built out, roughly in this order: specialist electronics retailers, department stores, beauty retailers, and home improvement, where location-dependent pricing makes the job more interesting than it looks.
If one of those is your target, say so — an in-progress source can usually be prioritised, and you get it sooner than the roadmap suggests.
Your site is not on the list
That is the normal case. Most projects we take on involve at least one source we have not touched before, and building a new parser is ordinary work rather than an exception.
Send the URLs, ideally down to the category pages rather than the homepage, along with the fields you need. Within a day or two you get back a tier assessment, a realistic coverage estimate, a fixed price and a delivery date. If the site is straightforward, a free test scrape of around 100 products comes with it.
Per project for a one-time extraction. Recurring monitoring is quoted per source and interval. Heavily defended sites are priced separately, and we say so before you ask.
What we will not take on
Anything behind a login or a paywall. Personal data — customer names, contact details, reviewer profiles. Review text for republication. Sites whose terms we would have to misrepresent ourselves to satisfy. We also decline projects where the brief only makes sense if the target does not find out. Saying this up front saves a conversation that would end the same way later.
Published output
Rather than describe our data quality, we publish full runs. Open one, check the field coverage, look at how the edge cases are handled.
Adding a source
How long before a new site is delivering data?
Two to four working days for a straightforward source once the brief is agreed. One to two weeks where the site has strong bot protection, location-dependent pricing or a deep variant structure. You get a date with the quote rather than after.
Can you do several sites in one project?
Yes, and comparing one category across three retailers usually answers more than a full catalog from one of them. Each source needs its own parser, so each is priced separately, but the schema and delivery are shared — the second and third site cost less than the first.
What happens when a site redesigns?
We rewrite the parser. On monitored feeds this is the largest source of ongoing work and it is covered by the subscription, so a redesign does not turn into an invoice or a gap in your data.
Why not just claim you can scrape any website?
Because it is the sort of claim that survives exactly until the first difficult site. Some sources are genuinely uneconomic to run at volume, and a client who finds that out after signing has been badly served. Listing what we run, and grading what we don't, is slower to sell and much easier to deliver.
What we do with these sources
Tell us which sites matter
Send the URLs and the fields you need. You will get a tier assessment, a price and a delivery date, with a free test scrape where the site allows it.