The Brutalist Report - cloudflare

Cloudflare Blog

Content Independence Day: no AI crawl without compensation! [18d]
The crawl before the fall… of referrals: understanding AI’s impact on content providers [18d]
Control content use for AI training with Cloudflare’s managed robots.txt and blocking for monetized content [18d]
Introducing pay per crawl: enabling content owners to charge AI crawlers for access [18d]
From Googlebot to GPTBot: who’s crawling your site in 2025 [18d]
Message Signatures are now part of our Verified Bots Program, simplifying bot authentication [18d]