Law.com Subscribers SAVE 30%

Call 855-808-4530 or email [email protected] to receive your discount on a new subscription.

Internet 'Data Scraping'

By Anthony J. Dreyer and Jamie Stockton
August 02, 2013

The proliferation of Internet access and mobile devices has led to an exponential explosion of content on the Web, creating a vast repository of “publicly available” information. This includes not only news, business and financial information, but also personal data, movie and restaurant reviews, concert ticket sales, flight information, and a virtually endless array of other categories. This same technological explosion, however, has made it far easier for third parties to extract this data for commercial sale and use ' and to do so for free and without authorization. This data extraction, commonly referred to as “scraping,” “crawling,” or “spidering” (collectively “scraping”), creates legal issues and concerns for both sides of this issue ' those who want to scrape, and those who want to protect against scraping of their websites. See, EF Cultural Travel BV v. Zefer, 318 F.3d 58, 60 (1st Cir. 2003) (“A scraper, also called a 'robot' or 'bot,' is nothing more than a computer program that accesses information contained in a succession of webpages stored on the accessed computer”); eBay v. Bidder's Edge, 100 F. Supp. 2d 1058, 1060 (N.D. Cal. 2000).

While it is possible to embed instructions on websites that inform the scraping software whether scraping is permitted (called “robot.txt” files), compliance with such instructions is voluntary. See, Bidder's Edge , at 1061.

Read These Next
The DOJ's Corporate Enforcement Policy: One Year Later Image

The DOJ's Criminal Division issued three declinations since the issuance of the revised CEP a year ago. Review of these cases gives insight into DOJ's implementation of the new policy in practice.

The DOJ's New Parameters for Evaluating Corporate Compliance Programs Image

The parameters set forth in the DOJ's memorandum have implications not only for the government's evaluation of compliance programs in the context of criminal charging decisions, but also for how defense counsel structure their conference-room advocacy seeking declinations or lesser sanctions in both criminal and civil investigations.

Use of Deferred Prosecution Agreements In White Collar Investigations Image

This article discusses the practical and policy reasons for the use of DPAs and NPAs in white-collar criminal investigations, and considers the NDAA's new reporting provision and its relationship with other efforts to enhance transparency in DOJ decision-making.

A Lawyer's System for Active Reading Image

Active reading comprises many daily tasks lawyers engage in, including highlighting, annotating, note taking, comparing and searching texts. It demands more than flipping or turning pages.

Bankruptcy Sales: Finding a Diamond In the Rough Image

There is no efficient market for the sale of bankruptcy assets. Inefficient markets yield a transactional drag, potentially dampening the ability of debtors and trustees to maximize value for creditors. This article identifies ways in which investors may more easily discover bankruptcy asset sales.