Skip to content
Hacker News · AI· titusblair·· 3 hours agoSelectedAI score69

METR unpacks Anthropic's reported 1.9x R&D speedup: Why the actual range spans 1.1x to 2.9x

METR's 1.9x AI R&D speedup at Anthropic, and why it ranges from 1.1x to 2.9x

AI guide

Independent evaluation organization METR released an assessment showing that Anthropic's internal model capability growth rate has reached roughly 1.9x its previous pace, with a subjective estimate attributing about 80% of this gain to AI itself. However, this estimate heavily depends on link-function assumptions, with speedups varying between 1.1x and 2.9x under different task difficulty distributions. While the evaluation spans unreleased internal models and fits a capability index, several core data points—including benchmark names—have been redacted by Anthropic.

Why it matters

This analysis breaks down the statistical assumptions and data blind spots behind model R&D acceleration assessments, offering an objective view of the measurement limitations surrounding frontier model self-improvement.

Source:Hacker News · AI · theainewsreport.com