JsonEngine · FAQ

Frequently asked questions

Everything clients ask before working with us — how the service runs, what we deliver, the legal side, and the technology behind it.

The essentials
What does JsonEngine do?

We collect data from public web sources and deliver it to you as a clean, structured feed. You tell us which sites and which fields matter — prices, stock availability, product catalogs, and more — and we hand back ready-to-use data, refreshed on a schedule that fits your business. No infrastructure to build and no scrapers to maintain on your side.

What kind of data can you collect?

Anything publicly visible on a website and useful to your business: product prices and price changes, stock and availability, full catalogs and assortments, product attributes, ratings and review counts, store listings, and corporate business information. We focus on publicly available data and do not collect personal data.

What types of websites can you scrape?

Any site with openly accessible data. Most clients ask us to monitor large online stores and marketplaces — that’s where we do the bulk of our work — but we also handle smaller retailers, brand sites, directories, news, and listings. If the data is public, we can usually collect it.

In what formats do I receive the data, and how?

You get the data in whatever format fits your workflow: CSV, XML, JSON, Excel, Google Sheets, SQL, or PDF. Delivery is just as flexible — pull it through our API, or access it directly in your browser through a cloud interface that works much like Dropbox or Google Drive. Many clients pipe it straight into their own systems.

How fresh is the data, and how often is it updated?

Freshness is the whole point. We deliver an up-to-date feed on a recurring schedule — daily for most clients — so you’re always working with current prices and availability rather than a stale one-off snapshot. The update frequency is set per project based on how fast your target data changes.

Which countries and regions do you cover?

Practically all of them. Demand for structured data is global, and we work with clients regardless of country or region — including the major markets in the US, UK, Canada, the EU, and Australia. Tell us which markets and sites you care about and we’ll cover them.

How do we get started?

You describe what you need — the sites, the fields, the format, and how often you want the feed. We scope the work, agree on the setup, and stand up the pipeline. From there you receive your data on the agreed schedule, and we handle monitoring and upkeep — including updating the parser whenever a site changes — behind the scenes.

Legal & ethics
Is web scraping legal?

We work only with publicly available data, and we don’t collect anything that infringes copyright or contains personal data. We operate within applicable laws and ethical standards, and we structure each project around those limits. Because regulations differ by jurisdiction and use case, we’re always happy to discuss the specifics of your project before we begin.

Do you collect personal data?

No. We deliberately stay away from personal data and focus on public, corporate, and product information — prices, availability, catalogs, business listings, and the like. When we build company databases, we use only publicly available corporate information.

Will scraping affect the target website’s performance?

We design our collection to minimize impact on the sites we visit. We run at a measured request frequency so we don’t create unnecessary load — the goal is to gather data quietly in the background, not to disrupt anyone’s operations.

How is scraping different from a DDoS attack?

They’re fundamentally different. A DDoS attack is designed to overwhelm a site and take it down. Scraping does the opposite: it collects data while keeping the site fully operational. We have every reason to keep our target sites healthy and responsive.

Do the websites know you collect their data?

Clients ask us this often. Our position is simple — we’re open to communication, and if a company has concerns or requests, we’re willing to work with them. Our priority is to deliver useful data to our clients while staying compliant with laws and regulations.

Technology & reliability
How do you handle anti-bot protection, Cloudflare, and captchas?

This is routine for us. We rotate proxies, vary digital browser fingerprints, and lean on years of experience to collect data reliably. When data sits behind images, we use image-recognition tools to read it accurately; when there’s a captcha, we use specialized solving services. Cloudflare and similar tools are built mainly to stop DDoS, not parsing — we’ve learned to work with them. Strong defenses add complexity, but at a professional level they don’t stop the process.

What technology and infrastructure do you use?

Our infrastructure is built to handle big data reliably. We combine modern cloud services with proven server solutions — our stack includes AWS (Lambda, EC2, SQS, DynamoDB, RDS), CI/CD pipelines, Docker, Node.js, and TypeScript, running on a fleet of high-core-count servers for fast, stable processing. This lets us collect at scale without sacrificing reliability.

What happens if a site changes and breaks the scraper?

Sites change, and occasionally an unexpected update can interrupt collection. We continuously monitor our pipelines and adapt quickly when something shifts, so disruptions are short and rare. If anything about the result isn’t right for you, we work with you to find a solution — but in practice, problems come up infrequently.

How long do you store my data?

We’re focused on relevance, not archiving. Data older than about 10 days is removed, since keeping long histories is resource-intensive and most clients care about what’s current. If you need longer retention, we can arrange it as part of your project.

Working with the data
What do clients typically do with the data?

Clients use it to make informed decisions — tracking competitor prices, monitoring availability, analyzing assortments, and spotting market trends. Many integrate the feed directly into their own systems, such as Power BI, Google BigQuery, or even Excel, so the data flows straight into their existing workflow.

Why order scraping instead of doing it in-house?

For some clients it’s about staying out of data collection entirely; for others it’s about saving the time and resources of hiring and managing developers to build and babysit scrapers. Reliable, ongoing parsing takes experience and a team that can keep it stable through constant site changes — and that’s exactly what we provide.

Can you build company databases or do product comparisons?

Yes. We regularly build databases of organizations from public information — for example, online stores in a given niche, auto repair shops, or medical companies — always limited to publicly available corporate data. We also have experience with semi-automatic product comparison and can take on that kind of work by arrangement.

Who’s on your team, and how much support does the service need?

We’re a focused team of skilled developers backed by a support team whose job is to keep your data accurate and consistently updated. We constantly monitor and adapt our processes to keep delivery uninterrupted, so the day-to-day upkeep stays on our side, not yours.

Still have a question?

Tell us the sites and data you need — we’ll scope it and reply with a plan.

Contact us