
Breakdown / AI research · Independent evaluation
METR sees faster AI research at Anthropic. Much of its evidence is redacted.
An outside assessment estimates roughly doubled progress on well-defined tasks. Its conclusions are public, but important parts of the evidence behind them aren't.
Anthropic appears to be improving its AI models on well-defined tasks at roughly twice its pace between March 2024 and October 2025, according to an outside assessment released November 5. Much of the evidence behind that conclusion cannot be made public.
METR, the nonprofit that conducted the assessment, believes AI assistance explains much of the current acceleration, with substantial uncertainty. It says Anthropic redacted a majority of the evidence it considered most informative. November 5 assessment
That leaves readers with a consequential judgment about AI helping to build better AI, and only part of the material needed to assess it.
What the estimate means
Twice the pace means making roughly two years of earlier progress in one year. It doesn't mean that every model is twice as intelligent, or that every customer's task takes half the time.
METR reviewed internal information and interviewed staff from August 12 to October 29. Staff described AI primarily assisting researchers. The assessment doesn't isolate AI's effect in a controlled experiment, and test gains may not carry over to messier work. Anthropic says the redactions protect confidential information that could accelerate competitors. Methods and limitations
This was a different access arrangement
Anthropic announced broader access for six METR staff on October 15. It described access similar to its own risk-assessment teams, with exceptions for categories such as customer information and model architecture. The company says it cannot veto publication under that arrangement, but can require specified kinds of information to be removed. Anthropic ultimately decides what falls within those categories. Anthropic's account
The November 5 assessment came from a separate, earlier engagement. It should not be read as a completed test of those newer access promises. Engagement details
An early warning is a separate judgment
METR also published a review of the automated-research section of Anthropic's August risk report. It agrees that the models covered as of July 15 posed low acute risk in that area. It also considers it plausible that an early-warning threshold had already been crossed, although both teams lean toward it not having been crossed. METR says the public report understated the acceleration then occurring. Risk-report review
A warning threshold is meant to prompt attention before a feared outcome arrives. Disagreement about that threshold can coexist with agreement that the immediate risk was low. Those historical findings also cannot settle the safety of models released later.
For the next assessment, access is only the first question. Can evaluators check a disputed result themselves? Can they describe the evidence that changed their minds? Broader access will be easier to judge when readers can see what it lets them explain.