Love that Cloudflare is enforcing the robots.txt rules with all crawlers (especially the AI training set ones), read more here; blog.cloudflare.com/ai-audit-enf... h/t @jono.id - ThreadSky

hellemans.me • 87 days ago

Love that Cloudflare is enforcing the robots.txt rules with all crawlers (especially the AI training set ones), read more here; https://blog.cloudflare.com/ai-audit-enforcing-robots-txt/ h/t @jono.id