Is Web Scraping Legal in the US in 2026?

Last reviewed: October 8, 2026 · 13 min read

Short answer: collecting publicly visible data from US websites is often lawful, and courts have repeatedly refused to treat it as hacking. But “legal” is not one question. It is at least four, and a project can be fine on one and exposed on another.

Why “is web scraping legal?” has no one-word answer

When people ask whether scraping is legal, they usually mean “can I get sued or prosecuted for this?” US law answers that through several unrelated bodies of law. A court can rule that you did not hack anyone and, in the same case, that you broke a contract. That is exactly what happened to hiQ Labs.

It helps to treat every scraping project as four separate questions:

The rest of this guide walks through each layer, then lists the cases that define the rules today and a checklist we use before starting a project.

Layer 1: Access and the CFAA after Van Buren and hiQ

The Computer Fraud and Abuse Act makes it a crime, and a civil claim, to access a computer “without authorization” or to “exceed authorized access.” For years, site owners argued that scraping against their wishes met that definition.

Two decisions narrowed it. In Van Buren v. United States (June 2021), the Supreme Court read the CFAA as a “gates-up-or-down” test: the question is whether you entered a part of a system that was closed to you, not whether you used data for a purpose the owner disliked. Then in April 2022 the Ninth Circuit, applying that logic in hiQ Labs v. LinkedIn, held that data on a public website, with no password gate, is unlikely to be “without authorization” under the CFAA.

Two caveats matter in practice. The hiQ ruling was about a preliminary injunction, not a final judgment, and it binds federal courts in the Ninth Circuit (California and eight other western states), not the whole country. And the Department of Justice’s 2022 CFAA charging policy, often quoted in this debate, protects good-faith security research. It is not a license for commercial data collection.

Logins and cease-and-desist letters change the picture

The gate in “gates-up-or-down” is usually a login. Once you scrape behind an account, especially after the site has told you in writing to stop, the CFAA analysis gets much less comfortable. Amazon v. Perplexity shows how quickly this moves. In March 2026 a federal judge in San Francisco granted Amazon a preliminary injunction against Perplexity’s Comet shopping agent, relying on the idea that a platform can revoke authorization even when its users consent. The Ninth Circuit later vacated that injunction, finding on the current record that the AI agent acted as the user’s tool rather than accessing Amazon itself.² The case is not over, and its facts (a logged-in AI agent buying on a user’s behalf) are far from ordinary price collection, but it shows where the line is being drawn: account areas, not public product pages.

The 2026 twist: anti-bot systems as DMCA “access controls”

Section 1201 of the DMCA forbids circumventing a “technological measure that effectively controls access” to a copyrighted work. Until recently few people connected that to scraping. In 2026 two courts did, with opposite results:

• Google v. SerpApi (N.D. Cal., July 20, 2026): the court dismissed Google’s DMCA suit against the search-results API provider.
• Reddit v. SerpApi (S.D.N.Y., July 2026): the court let Reddit’s circumvention claims proceed, treating Google’s SearchGuard bot defense as an access control even though humans can read the same pages. According to a summary by Loeb & Loeb, the court distinguished the Google case because Reddit had plausibly authorized that protection for its content.

The practical lesson does not depend on which court is right. Getting around a CAPTCHA or JavaScript challenge is no longer just an engineering question. If a site has put up a technical barrier, defeating it can be cited against you.

Damages have a floor

Civil CFAA claims require at least $5,000 in “loss.” In Ryanair v. Booking.com, a Delaware jury found Booking liable in 2024 for exactly that amount, and in January 2025 the judge threw the verdict out because Ryanair had not proven $5,000 of qualifying loss. Polite, low-volume scraping that causes no measurable harm is a weak target for a CFAA suit. A scraper that knocks a site over is not.

Layer 2: Contracts and terms of service

Almost every retailer’s terms forbid automated collection. Whether those terms bind you depends on whether you agreed to them, and courts look at how you agreed.

• Clickwrap: you created an account and clicked “I agree.” Courts usually enforce these. Scraping while logged in to that account is the riskiest contract scenario.
• Browsewrap: the terms are a link in the footer and you never clicked anything. Courts enforce these far less often, because there is no clear assent.

The cases line up with that distinction. hiQ won its CFAA argument but, in November 2022, the district court found it had breached LinkedIn’s user agreement, partly through fake accounts. The case ended in December 2022 with a consent judgment: $500,000, deletion of the scraped data and the code, and a permanent ban on scraping LinkedIn. In January 2024 the opposite happened in Meta v. Bright Data: Judge Edward Chen found that Bright Data’s logged-off scraping of public Facebook and Instagram pages did not breach Meta’s terms, because Bright Data was not “using” the services as a logged-in user. Meta dropped its remaining claim a month later. In May 2024 a different judge in the same court dismissed most of X Corp’s contract claims against Bright Data, reasoning that a site cannot use contract law to create a copyright-like monopoly over public data.

Contracts are also where the newest fights are. In September 2026 a San Francisco Superior Court judge let Reddit’s breach-of-contract, interference and unfair-competition claims against Anthropic go forward.³ That is a ruling on whether the claims can proceed, not a finding of liability. Still, the direction is clear: well-drafted terms plus documented notice to the scraper give site owners claims that survive early motions.

Layer 3: Copyright and what you copy

Facts are not copyrightable in the US. The Supreme Court said so in Feist v. Rural Telephone (1991) about a phone book, and the logic covers most of what e-commerce scraping collects: prices, stock status, SKUs, GTINs, dimensions, ratings counts. A price list is a set of facts.

Creative expression is different. Product descriptions written by a brand’s copywriter, lifestyle photography and the text of customer reviews can all be protected. Copying them into a database you use internally for analysis is a much weaker infringement story than republishing them on your own site. Courts also use copyright the other way: in X Corp v. Bright Data, the judge held that contract claims trying to control public data were preempted by copyright law.

A simple rule works for most commercial projects: collect facts, analyze text, don’t republish creative content you did not write.

Layer 4: Personal data and privacy laws

Here “public” helps less than people expect. About twenty US states now have comprehensive consumer privacy laws, with Indiana, Kentucky and Rhode Island taking effect on January 1, 2026.⁴ California’s CCPA excludes some “publicly available” information, but the definition is narrow and does not cover everything you can see in a browser. If EU residents appear in the data, the GDPR applies regardless of where you are.

For price and catalog work the fix is easy, because products are not people. The exposure comes from reviews, Q&A sections and marketplace seller profiles, which contain names, usernames and sometimes locations. When we run review collection projects, reviewer identifiers are dropped or hashed at extraction time, before anything is stored.

Does robots.txt make scraping illegal?

No, not by itself. The Robots Exclusion Protocol is now a formal standard, RFC 9309, and the standard itself says its rules are not a form of access authorization. A robots.txt file is a request, not a lock.

It still matters. It is evidence of what the site owner wants, it shows up in disputes, and ignoring it weakens a good-faith argument. Walmart’s robots.txt, for example, disallows /search for all crawlers. We treat robots.txt as a strong signal for what to collect and how fast, even though breaking it is not automatically unlawful.

The cases that define the rules in 2026

A practical checklist before you scrape

This is the checklist we run before accepting a price monitoring or catalog project. None of it guarantees anything, but each item removes a common way scraping disputes start.

1. Public pages only. No accounts, no logins, no paywalled content.
2. No circumvention. If a page is behind a CAPTCHA or challenge that a normal visitor would also face, the answer is to slow down or use an official channel, not to defeat it.
3. Facts, not creative content. Prices, availability, identifiers and specs. Descriptions and images only for internal analysis, never republished.
4. Strip personal data at the source. Reviewer names, usernames and profile links don’t enter storage.
5. Be gentle. Rate limits per domain, off-peak scheduling, caching. A site that never notices you never sends a letter.
6. Read robots.txt and the terms, even though browsewrap terms are weak. Know what you’re deciding to ignore and why.
7. Prefer an official API when it covers the use case. We compare the two in Web scraping vs official APIs.
8. Stop when asked and get advice. A cease-and-desist letter turns ordinary scraping into the scenario courts treat most harshly.
9. Keep records: what you collected, when, from where, and under which rules.

Some targets deserve extra care because their owners litigate. Amazon is one; we cover the technical side in our Amazon reality check.

FAQ