Competitor price monitoring
A repeating readout of what your competitors charge, taken at the interval you choose and delivered as a file you can compare against your own catalog. You see the moves; we handle the scraping, the proxies and the parser upkeep behind them.
Manual price checks stop working around the third competitor
Checking a handful of SKUs by hand is fine for a week. Once you are watching several hundred products across three or four stores, the spreadsheet goes stale faster than anyone can refresh it, and the interesting events — a rival dropping 20% on your best-selling line, a supplier going out of stock — get noticed days late.
The alternative most teams try next is building a scraper in-house. That works until the target site rolls out a redesign, adds a bot check, or starts serving different prices depending on the visitor's location. Then someone on your team owns a maintenance job nobody planned for.
This service exists to take that job off your plate and hand back a file that is correct on the morning you open it.
What each row contains
A standard export looks like this. Fields can be added or dropped during scoping — these are the ones almost every project ends up using.
| Field | Example | Why it matters |
|---|---|---|
| product_url | https://…/p/12345 | The canonical link for matching and verification |
| sku | GAP-4471-BLK-M | Retailer's own identifier, used to track a product across runs |
| title | Slim-fit cotton shirt | Needed when matching products across different stores |
| brand | Gap | Brand-level price comparison and share of shelf |
| price | 39.95 | The price a shopper actually pays right now |
| list_price | 59.95 | Reference price, where the retailer publishes one |
| discount_pct | 33.4 | Discount depth, calculated so you do not have to |
| currency | USD | Matters as soon as more than one region is involved |
| in_stock | true | An out-of-stock competitor is a pricing opportunity |
| seller | Marketplace seller name | On marketplaces, one product can carry many offers |
| category_path | Men > Shirts > Casual | Lets you roll prices up to category averages |
| collected_at | 2026-09-08T06:00:12Z | Every reading is timestamped, so history is reconstructable |
How often it runs is a business decision, not a technical one
Faster is not automatically better. Higher frequency means more requests, more proxy cost and more noise in the data. We usually settle on one of three patterns:
- Weekly — enough for range benchmarking and quarterly pricing reviews. The cheapest option and the right default for slow-moving categories like furniture.
- Daily — the standard choice for apparel, beauty and general retail, where promotions turn over on a weekly cycle and you want to catch them the morning they start.
- Hourly — for electronics, marketplace repricing and any category where a competitor's algorithm moves prices during the day.
We can also run mixed schedules: hourly on a watchlist of key SKUs, daily on the rest of the catalog. That keeps the cost proportional to how much each product actually matters.
Three things that quietly ruin price data
Location-dependent pricing
Grocery chains and home improvement retailers price by store. Home Depot and Kroger will happily show prices to a scraper that has not set a zip code — they just will not be the prices any real customer sees. We pin a location per feed and hold it across every run, so your comparisons stay against a consistent market.
Variants and multiple sellers
On a marketplace, one product page can carry a dozen offers from different sellers at different prices, and a fashion product can exist as forty colour and size combinations. Collapsing that into a single row loses the signal; keeping all of it floods the file. We agree the granularity during scoping — usually parent SKU with the winning offer, plus a separate variant-level export when you need it.
Pagination caps
Most large retailers stop paginating somewhere between 25 and 250 pages, and some cap search results at 10,000 items. A naive crawler quietly collects the first slice and reports success. We split large categories by price band, brand or subcategory until each slice fits under the cap, then reassemble. It is unglamorous work and it is the difference between a complete catalog and a plausible-looking fragment.
Getting a feed live
Send the targets
Competitor URLs, the categories that matter and roughly how many products are involved.
Free test scrape
Around 100 products from your real targets, so you can check the fields before agreeing anything.
Set the schedule
Interval, delivery format and destination — inbox, Sheets, S3 or an API endpoint.
It runs
Data arrives on schedule. When a site changes, we fix the parser without a change request.
What it costs
Monitoring is quoted per source and interval, since the effort depends far more on how a site defends itself than on how many rows come out. Send the targets and you will get a fixed monthly number back.
Price monitoring, in practice
How do you match my products to a competitor's?
Where a shared identifier exists — GTIN, UPC, MPN, or a brand and model number — we match on that, which is exact. Where it does not, as in most apparel and private label, we match on brand plus normalised title plus attributes and give you a confidence score per pair. You review the low-confidence matches once, we lock them in, and subsequent runs reuse the mapping.
Will the competitor know they are being monitored?
Our traffic is a small fraction of a large retailer's daily volume and is spread across rotating addresses at a request rate below what a busy human shopper generates. We collect only pages any visitor can open. That said, we do not promise undetectability, and we will not take on a project whose premise is deception.
Can you alert me when a price changes?
Yes. Change detection between runs is standard, and alerts can be pushed by email or webhook when a monitored SKU crosses a threshold you define — a drop over 10%, a competitor undercutting you, a product going out of stock.
What if a competitor blocks you mid-contract?
It happens, usually after a site upgrades its bot protection. We rework the approach at our cost. If a source becomes genuinely unworkable, we tell you plainly rather than quietly delivering degraded data, and we adjust the subscription.
How far back does price history go?
From the day your feed starts. We do not have retroactive history for sites you have not been monitoring, and neither does anyone selling it to you without caveats. If you need a baseline quickly, a one-time extraction across a broad category is the usual starting point.
Other services
Start with a free test scrape
Send two or three competitor URLs and the fields you care about. You will get real rows back from your own targets before any commitment.