The downfall of AI will be when AI generated content becomes greater than real content. Nothing like making copies of copies to reduce quality and resolution.
Seems to be inspired by some open source tools. Though would be better if cloudflare feed the bots junk instead so their operators learn to respect robots.txt
The AI company should pay the publisher/website to access their quality content; otherwise it’s just theft and I could care less what their scrapers get served.
Not to mention the publisher has to foot the bill of serving the scraper traffic… so respect the robots.txt file, it’s simple
Ideally, sure, but they don't respect the robots.txt. It's always been an honor system, so you can either keep fighting a pointless battle you can't win, or do something useful about it.
In the very last sentence without much of the story behind the origins. So I linked an article to provide the origin story in detail. If you don’t want that, you can simply scroll
Comments
https://www.404media.co/developer-creates-infinite-maze-to-trap-ai-crawlers-in/
Not to mention the publisher has to foot the bill of serving the scraper traffic… so respect the robots.txt file, it’s simple