Law.com Subscribers SAVE 30%

Call 855-808-4530 or email [email protected] to receive your discount on a new subscription.

Internet 'Data Scraping'

By Anthony J. Dreyer and Jamie Stockton
August 02, 2013

The proliferation of Internet access and mobile devices has led to an exponential explosion of content on the Web, creating a vast repository of “publicly available” information. This includes not only news, business and financial information, but also personal data, movie and restaurant reviews, concert ticket sales, flight information, and a virtually endless array of other categories. This same technological explosion, however, has made it far easier for third parties to extract this data for commercial sale and use ' and to do so for free and without authorization. This data extraction, commonly referred to as “scraping,” “crawling,” or “spidering” (collectively “scraping”), creates legal issues and concerns for both sides of this issue ' those who want to scrape, and those who want to protect against scraping of their websites. See, EF Cultural Travel BV v. Zefer, 318 F.3d 58, 60 (1st Cir. 2003) (“A scraper, also called a 'robot' or 'bot,' is nothing more than a computer program that accesses information contained in a succession of webpages stored on the accessed computer”); eBay v. Bidder's Edge, 100 F. Supp. 2d 1058, 1060 (N.D. Cal. 2000).

While it is possible to embed instructions on websites that inform the scraping software whether scraping is permitted (called “robot.txt” files), compliance with such instructions is voluntary. See, Bidder's Edge , at 1061.

Read These Next
New York's Latest Cybersecurity Commitment Image

On Aug. 9, 2023, Gov. Kathy Hochul introduced New York's inaugural comprehensive cybersecurity strategy. In sum, the plan aims to update government networks, bolster county-level digital defenses, and regulate critical infrastructure.

The Bankruptcy Hotline Image

Recent cases of importance to your practice.

The DOJ's Corporate Enforcement Policy: One Year Later Image

The DOJ's Criminal Division issued three declinations since the issuance of the revised CEP a year ago. Review of these cases gives insight into DOJ's implementation of the new policy in practice.

How AI Has Affected PR Image

When we consider how the use of AI affects legal PR and communications, we have to look at it as an industrywide global phenomenon. A recent online conference provided an overview of the latest AI trends in public relations, and specifically, the impact of AI on communications. Here are some of the key points and takeaways from several of the speakers, who provided current best practices, tips, concerns and case studies.

Use of Deferred Prosecution Agreements In White Collar Investigations Image

This article discusses the practical and policy reasons for the use of DPAs and NPAs in white-collar criminal investigations, and considers the NDAA's new reporting provision and its relationship with other efforts to enhance transparency in DOJ decision-making.