Cloudflare's AI Crawler Rules May Block Googlebot by Default
Search Engine Journal12h ago·1 min readSEO & Search
AI Summary
Cloudflare has introduced new AI crawler rules that allow site owners to manage how AI bots access their content. Default settings, effective September 15th, could inadvertently block Googlebot if sites are configured to block training data collection.
⚡ Marketer Insight
AI's increasing role in web crawling presents a new technical hurdle for SEO: Cloudflare's default AI crawler settings could unintentionally disrupt Googlebot's access, impacting search visibility if not proactively managed.
#cloudflare#googlebot#seo#ai crawlers
Original article
Search Engine Journal