METR tests Claude Opus 5.5: a modest step up, unlikely to fully automate AI research
Testing it before release, METR found Claude Opus 5.5 better than Claude Fable 5.1 but no big leap, and still weak in judgement.
METR, an independent evaluator, tested Claude Opus 5.5 for ten business days before Anthropic released it. It used five research-style tasks, from speeding up a small model’s training to writing an open-ended research report. Some tasks had results that are easy to check, others did not.
Opus 5.5 did better than Claude Fable 5.1 on both kinds. METR called the gains incremental rather than a sudden jump, and said weaknesses in judgement remain. It concluded that Opus 5.5 is “unlikely to be able to fully automate AI R&D”.
A separate METR team estimated that AI was speeding up Anthropic’s progress by about 1.5×, with perhaps a 30% chance of 2×. That was a preliminary figure. Six weeks later, METR’s fuller audit inside Anthropic raised it to about 1.9×.
What it means
Opus 5.5 is a better research assistant than the model before it, not a replacement for AI researchers. The bigger finding came later: METR’s audit found AI speeding up Anthropic’s progress more than this first estimate suggested.
Caveats
- The text comes from Anthropic’s system card (the safety report published with a model), which Anthropic could edit and METR signed off on.
- METR did not assess alignment (whether the model pursues the goals its makers intend) and reported no time-horizon number. It used five custom tasks instead.