EFF: bills to unmask 'stealth crawlers' threaten the open web more than the crawlers do
EFF — Deeplinks · Tori Noble · source ↗
The EFF pushes back on a new legislative boogeyman. “Stealth crawlers,” its Deeplinks post argues, are simply automated tools that collect public web data without disclosing the operator’s identity — and anonymous crawling underwrites investigative journalism (The Markup’s and ProPublica’s Amazon investigations), academic research, security monitoring, and privacy tools like EFF’s own Privacy Badger.
The trigger is real legislation. New York’s legislature has passed the NY Stealth Crawler Protection Act (S9934), now on Governor Hochul’s desk; it would make crawling news sites illegal without revealing who operates the crawler and every future use of the data, and would let publishers obtain court orders unmasking a crawler with no evidence of wrongdoing. EFF expects copycats in other states and, eventually, Congress.
Strip away the AI framing and this is an identity fight: whether anonymous automated access to public information survives, or whether “who is your bot” becomes a precondition for reading a public page. The same instinct — bind a verified identity to every actor on the wire — runs straight through the caller-authentication debates on the other side of this beat. EFF’s counter is the right one: the harm is overaggressive crawling, and you fix load with rate limits, not by outlawing anonymity.