I wonder which bots weren’t following the rules and will actually be stopped by this. - apart from all the scrapers faking that they are said bots.
Reposted from Jono Alderson
Cloudflare now enforcing your robots.txt rules. Love it.

blog.cloudflare.com/ai-audit-enf...

Comments