Hacker News · AI· twapi·· 8 hours agoSelectedAI score74
Google, OpenAI, Microsoft, and Anthropic Jointly Back Unified robots.txt AI Training Directive
Google, OpenAI, and Anthropic back a shared robots.txt AI training signal
AI guide
Google, OpenAI, Microsoft, and Anthropic announced joint support for a new robots.txt directive, `__usage_ai-training`, enabling websites to declare AI training preferences across major model providers via a single User-Agent rule instead of separately configuring Google-Extended, GPTBot, and Claudebot.
Why it matters
A unified robots.txt directive across the four major providers simplifies opt-out configuration for websites, though implementation timelines vary and compliance lacks enforcement mechanisms.
Source:Hacker News · AI · coywolf.com