The legality of web scraping varies across the world. In general, web scraping may be against the
terms of service of some websites, but the enforceability of these terms is unclear. However, the effectiveness of these claims relies upon meeting various criteria, and the case law is still evolving. For example, with regard to copyright, while outright duplication of original expression will in many cases be illegal, in the United States the courts ruled in
Feist Publications v. Rural Telephone Service that duplication of facts is allowable. U.S. courts have acknowledged that users of "scrapers" or "robots" may be held liable for committing
trespass to chattels, In 2012, a startup called 3Taps scraped classified housing ads from Craigslist. Craigslist sent 3Taps a cease-and-desist letter and blocked their IP addresses and later sued, in
Craigslist v. 3Taps. The court held that the cease-and-desist letter and IP blocking was sufficient for Craigslist to properly claim that 3Taps had violated the
Computer Fraud and Abuse Act (CFAA). Although these are early scraping decisions, and the theories of liability are not uniform, it is difficult to ignore a pattern emerging that the courts are prepared to protect proprietary content on commercial sites from uses which are undesirable to the owners of such sites. However, the degree of protection for such content is not settled and will depend on the type of access made by the scraper, the amount of information accessed and copied, the degree to which the access adversely affects the site owner's system and the types and manner of prohibitions on such conduct. In a 2014 case, filed in the
United States District Court for the Eastern District of Pennsylvania, e-commerce site
QVC objected to the Pinterest-like shopping aggregator Resultly's 'scraping of QVC's site for real-time pricing data. QVC alleges that Resultly "excessively crawled" QVC's retail site (allegedly sending 200-300 search requests to QVC's website per minute, sometimes to up to 36,000 requests per minute) which caused QVC's site to crash for two days, resulting in lost sales for QVC. QVC's complaint alleges that the defendant disguised its web crawler to mask its source IP address and thus prevented QVC from quickly repairing the problem. This is a particularly interesting scraping case because QVC is seeking damages for the unavailability of their website, which QVC claims was caused by Resultly. In the plaintiff's web site during the period of this trial, the terms of use link are displayed among all the links of the site, at the bottom of the page as most sites on the internet. This ruling contradicts the Irish ruling described below. The court also rejected the plaintiff's argument that the browse-wrap restrictions were enforceable in view of Virginia's adoption of the Uniform Computer Information Transactions Act (UCITA)—a uniform law that many believed was in favor on common browse-wrap contracting practices. In
Facebook, Inc. v. Power Ventures, Inc., a district court ruled in 2012 that Power Ventures could not scrape Facebook pages on behalf of a Facebook user. The case is on appeal, and the
Electronic Frontier Foundation filed a brief in 2015 asking that it be overturned. In
Associated Press v. Meltwater U.S. Holdings, Inc., a court in the US held Meltwater liable for scraping and republishing news information from the Associated Press, but a court in the United Kingdom held in favor of Meltwater. The
Ninth Circuit ruled in 2019 that web scraping did not violate the CFAA in
hiQ Labs v. LinkedIn. The case was appealed to the
United States Supreme Court, which returned the case to the Ninth Circuit to reconsider the case in light of the 2021 Supreme Court decision in
Van Buren v. United States which narrowed the applicability of the CFAA. On this review, the Ninth Circuit upheld their prior decision.
Internet Archive collects and distributes a significant number of publicly available web pages without being considered to be in violation of copyright laws.
European Union In February 2006, the
Danish Maritime and Commercial Court (Copenhagen) ruled that systematic crawling, indexing, and deep linking by portal site of real estate site does not conflict with Danish law or the database directive of the European Union. Ethical data scraping supports offmarket sourcing in business but must comply with GDPR to avoid privacy violations in automated data collection. In a February 2010 case complicated by matters of jurisdiction, Ireland's High Court delivered a verdict that illustrates the
inchoate state of developing case law. In the case of
Ryanair Ltd v Billigfluege.de GmbH, Ireland's High Court ruled
Ryanair's "
click-wrap" agreement to be legally binding. In contrast to the findings of the United States District Court Eastern District of Virginia and those of the Danish Maritime and Commercial Court, Justice
Michael Hanna ruled that the hyperlink to Ryanair's terms and conditions was plainly visible, and that placing the onus on the user to agree to terms and conditions in order to gain access to online services is sufficient to comprise a contractual relationship. The CNIL guidelines made it clear that publicly available data is still personal data and cannot be repurposed without the knowledge of the person to whom that data belongs.
Australia In Australia, the
Spam Act 2003 outlaws some forms of web harvesting, although this only applies to email addresses.
India Leaving a few cases dealing with IPR infringement, Indian courts have not expressly ruled on the legality of web scraping. However, since all common forms of electronic contracts are enforceable in India, violating the terms of use prohibiting data scraping will be a violation of the contract law. It will also violate the
Information Technology Act, 2000, which penalizes unauthorized access to a computer resource or extracting data from a computer resource. == Methods to prevent web scraping ==