News Publishers Block AI Crawlers by Default
Search Engine Journal · Jun 10 · 1 min read
What happenedMajor news outlets like Reuters and Time are now blocking AI bots by default, only allowing approved crawlers access to their content. This move by publishers aims to add friction to AI content scraping and protect their intellectual property.
Why it matters for marketersAs publishers increasingly restrict AI access, marketers relying on scraped web data for AI model training or competitive analysis will face significant hurdles. This signals a growing battle over data access and AI's insatiable appetite for content.
Topics: ai content, data scraping, publisher strategy, ai ethics