Goliath Super Intelligence
IndustryOctober 18, 20262 min read

Meta Strips Clause Allowing Unrestricted Scraping of News Content for AI Training

The company removed a term that had let it harvest any website material, including news articles, to feed its artificial-intelligence systems, after criticism from scholars and policy groups.

Meta announced that it has excised a specific provision from its commercial terms that previously granted the company the ability to draw on any material found on external domains, web pages or URLs for use across its product lineup. The amendment was posted on the company’s policy page and is now reflected in the updated terms of service.

The removed clause had stated that Meta could employ any content from such sources to provide, develop, improve, research, train, and preserve the integrity of its services and features. By embedding that language, the agreement effectively gave Meta a blanket right to incorporate third-party data into its platforms without separate licensing arrangements.

Legal analysts at Neiman Lab and Cornell digital-law professor James Grimmelmann argued that the provision enabled Meta to harvest news articles and other copyrighted material without respecting a site’s robots.txt directives. They contended that the wording superseded standard web-crawling restrictions, allowing the company to collect and reuse content for artificial-intelligence training regardless of publisher preferences.

Critics warned that the clause could have let Meta feed its AI models with scraped news content at no cost to the original publishers, raising questions about equitable compensation and control over editorial material. The ability to bypass blocking instructions was seen as a potential threat to the revenue streams of news organizations that rely on licensing fees.

Following sustained pushback from the policy community, Meta removed the contentious language but did not disclose any replacement terms governing data collection for AI development. The company’s statement emphasized a commitment to responsible data practices, yet it left open how future training data will be sourced, suggesting that new licensing frameworks may be negotiated with content owners.

The amendment may reshape the data pipeline feeding Meta’s large language models and other AI-driven services, which have previously relied on large-scale web scrapes. Without the broad permission previously granted, the firm is likely to seek explicit agreements with news outlets or to rely on publicly licensed datasets, potentially altering the cost structure of model training.

Industry observers note that Meta’s revision could set a precedent for other technology firms facing scrutiny over the use of publicly available web content in AI research. The change arrives amid broader debates about digital labor, copyright enforcement, and the ethical sourcing of training data, highlighting the growing regulatory attention on how AI systems are built.

Sources

  1. Meta removes language that may have allowed it to scrape news sites The Verge

More reports

Industry · October 18, 2026 · 1 min

Anthropic reports fewer junior hires and rising meeting time as AI reshapes work

Anthropic's October 2026 report shows junior technical staff dropped from 21 percent to 14 percent and that meeting time rose as Claude's text output increased.

Industry · October 18, 2026 · 1 min

Replit switches default AI model to OpenAI, citing cost and performance

At TechCrunch Disrupt, Replit CEO Amjad Masad announced OpenAI's ChatGPT will be the platform's default model, while Anthropic remains optional and enterprise revenue is set to exceed half by 2027.

Industry · October 18, 2026 · 1 min

Websites Push Back Against AI Agents From OpenAI, Meta, and Google

Major online services including Yelp, Delta, JPMorgan Chase, and Amazon are restricting autonomous agents, citing data licensing fees, security concerns, and ad revenue protection.