Cloudflare's AI Training Block: Googlebot Exempt, Bing Delayed
Semrush Blog · Sep 18 · 1 min read
What happenedCloudflare's new 'Disallow AI Training' setting allows users to block AI training crawlers from accessing their sites, but notably exempts Googlebot. Bing has announced it will honor this setting starting in early 2027.
Why it matters for marketersWhile Cloudflare offers a shield against unauthorized AI data scraping, the exemption for Googlebot highlights the ongoing tension between AI development and data privacy. Marketers must understand which AI crawlers are being blocked and which are still accessing their content, especially with Bing's delayed compliance.
Topics: ai training, data privacy, googlebot, bing