Block on Trump's Asylum Ban Upheld by Supreme Court
If you get a creepy, crawly feeling as your server slows down, it may be the robots scraping your website for information.
And if you get that nauseous, pit-of-your-stomach ache, maybe it's the legal bills you sense coming on. That's because it will take lawsuits or legislation to catch up with web scrapers, and the future is not certain.
Courts allow some companies to scrape websites, but not others. It depends in part on how judges apply old principles to new technology.
Robots already make up 46 percent of web traffic, and many companies use them to scrape information off the web. They do it for content, research, price comparison, weather data monitoring, and information.
The industries most affected by web scraping include real estate, digital publishing, e-commerce, directories, airlines, and travel. According to reports, web scraping is easy to carry out, but poses significant risks to businesses and individuals.
Meanwhile, the courts are sorting out the issues arising from the technology -- First Amendment, privacy rights, copyright, trademark and more. In a Pennsylvania case, a district court ruled against QVC when the company sued a web scraper for allegedly crashing its website. In another jurisdiction, an appeals court said a web scraper caused no damage in scraping public records.
"[O]ne potential trend from these and previous cases is that courts are less willing to rule in favor of website owners where there is no enforceable contract prohibiting scraping, particularly where the scraping isn't competitive," attorneys James Snell and Nicolar Mendaldo wrote for Bloomberg BNA.
In a closely watched case, LinkedIn is suing in San Francisco's federal court to protect its information from webscraping. LinkedIn claims HiQ, which collects information on employees for its clients, uses robots to bypass security measures and scrape data from LinkedIn's website.
At a hearing for a preliminary injunction, LinkedIn attorney Donald Verrilli said HiQ uses the scraped information against LindedIn members.
"More than 10,000 of our members work for employers that are HiQ clients," Verrilli said at hearing for a preliminary injunction. "HiQ is ratting them out to their employers, broadcasting the very information they don't want broadcasted."
Laurence Tribe, the constitutional scholar, said HiQ has a First Amendment right to gather the information. "To choke off speech and the precursor of speech, the gathering of facts and the analysis of information, is a dangerous path down which we should not go," Tribe said.
Judge Edward Chen said the case has "serious implications" for privacy rights and freedom to access public information on the web.