METR: Anthropic's AI research is running about 2x faster, but most evidence is redacted

METR, the AI evaluation nonprofit, published two documents on 5 November 2026 about how fast Anthropic's own AI systems are speeding up Anthropic's AI research. Its headline estimate: Anthropic is probably improving capabilities on well-specified tasks at about twice its March 2024 to October 2025 pace. METR also says that most of the evidence behind that conclusion was redacted by Anthropic, so readers have to trust METR's judgment more than METR itself would like. The report matters because Anthropic's responsible scaling policy treats a doubling of progress that can be attributed to AI as a formal threshold, and Anthropic told investors days earlier that it "may imminently" meet it.

Key points

• METR's assessment says Anthropic is likely improving capabilities on well-specified tasks at about 2x its March 2024 to October 2025 rate, and it would not be surprised by anything from 1.3x to 3x.
• On Anthropic's internal version of the Epoch Capabilities Index (AECI), METR estimates growth since October 2025 at about 1.9x the prior rate (95% confidence interval 1.5x to 2.8x).
• METR estimates that roughly 80% of the acceleration as of October 2026 is due to AI, which it calls a subjective and highly uncertain judgment.
• In a separate review of Anthropic's August 2026 Risk Report, METR agrees that catastrophic risk from models as of 15 July was low, but says the report overstates its confidence that the 2x threshold had not been crossed.
• Anthropic had unilateral rights to redact, and METR says most of the evidence it found most informative was redacted.

What happened

METR published two pieces on 5 November. The first is a review of the "Risks from automated R&D" section of Anthropic's August 2026 Risk Report. The second is a companion report, a "holistic assessment" of AI R&D acceleration at Anthropic. It comes from a high-access evaluation that ran from 12 August to 29 October 2026, was extended from four weeks to eleven, and was submitted for clearance on 3 November.

METR says the work was done by about 2.5 full-time-equivalent staff, with four primary authors. Anthropic gave them internal benchmark data, algorithmic-efficiency and compute metrics, headcount, code and pull-request data, AI spending, staff surveys and about 15 staff interviews. METR had no query access to unreleased models and did not sample them. It worked on Anthropic-issued devices using Anthropic-provided free tokens, and took no payment.

The review of the Risk Report is more guarded. Anthropic's report rates the overall risk as "Low" and concludes that neither of its two policy criteria had been met: the "Full Substitution" threshold (a model that can replace Anthropic's research scientists and engineers at competitive cost, within a factor of five) and the "2x Acceleration" threshold (a doubling of the rate of progress, likely from AI R&D automation, relative to both the expected rate and the fastest sustained rate seen before AI contributed much). Anthropic also said it was less confident than before, because its most concrete task evaluations have saturated and it sees early signs of acceleration.

METR agrees on the low catastrophic risk. On the 2x question it says it is plausible the models had already crossed the early-warning threshold by 15 July, although "most METR and Anthropic researchers lean toward it not being crossed." It says the public report overstates confidence and understates the evidence for acceleration.

Background

Anthropic's Responsible Scaling Policy, version 3.4, which took effect in July 2026, defines "dramatic acceleration" as progress running at twice the fastest sustained rate before AI was contributing much to AI R&D, plausibly because AI is automating AI R&D rather than because of more compute or other inputs. We reported on Anthropic's S-1 filing and its warning that it may imminently hit this threshold; the amended filing was dated 2 November, according to coverage of it. The two METR documents are an outside look at the same question.

This is also not METR's first review of an Anthropic risk report. Its blog lists reviews of the February 2026 Risk Report (8 May) and the August report's section on misalignment in high-stakes settings (20 October). METR has also red-teamed Anthropic's and OpenAI's internal agent-monitoring systems; we covered the OpenAI one.

The numbers

The assessment gives several estimates that point the same way but with wide uncertainty:

• Overall pace: about 2x the March 2024 to October 2025 rate on well-specified tasks, with a plausible range of 1.3x to 3x.
• AECI growth after October 2025: about 1.9x the earlier rate, with a 95% interval of 1.5x to 2.8x. The breakpoint it finds is 7 October 2025. Robustness checks give 1.1x to 2.9x depending mainly on the choice of link function, and including non-Anthropic frontier models gives about 1.4x.
• Trend shape: a permanent change in slope fits better than a one-time jump at the Mythos Preview model, in about 81% of bootstrap draws.
• Staff-reported uplift: informal Slack polls rose from about 1.2x in early 2025 to about 12x for the top internal model in October 2026. METR calls this very loose evidence.
• Benchmarks: the top internal model scores above 50% on all 15 benchmarks used, which raises concerns about measurement. METR also notes that token budgets on benchmarks may be too low to bring out the models' full ability, which could understate scores.

On the staff interviews, METR says responses centred around roughly 2x. Its own classification, done with Claude, of whether each interviewee thought the 2x threshold had been met:

Borderline or unclear4

Leans yes3

Yes2

No2

Leans no0

Anthropic staff interviews: has progress doubled? (METR's classification) (interviewees)

	interviewees
Borderline or unclear	4
Leans yes	3
Yes	2
No	2
Leans no	0

Source: METR, holistic assessment of AI R&D acceleration at Anthropic (5 Nov 2026)

For the review of the Risk Report, METR also did its own arithmetic. The private report's figures imply about 1.97x on the leading indicators, METR says; that figure is METR's calculation and does not appear in the private report. It argues that "less than a factor of 2", the public wording, would have been better stated as "around 2".

How the argument works

Two terms help. A compute multiplier is an algorithmic or architectural improvement that makes training more efficient, so the same compute buys more capability. Overall capability ("effective compute") is multipliers times raw training compute. METR says it is confident the acceleration was not driven by faster growth in training compute, though its supporting detail is redacted.

The second term is attribution. METR estimates about 80% of the October 2026 acceleration is due to AI and says non-AI factors probably contributed to the late-2025 jump. It is a judgment, and it feeds the policy question of whether the speed-up is the kind the policy cares about.

Ajeya Cotra, who works at METR but says she did not work on the project or see the unredacted report, argued in a post on 6 November that the numbers imply large uplift for researchers. Her reasoning is that if overall progress doubled, researchers' own speed-up has to be much higher, since labour is only one input alongside experiment compute. She estimates about 10x today, close to the staff self-reports, and reads the redacted evidence as suggesting acceleration may already exceed 2x. These are her opinions, not METR's, and she says her model depends on assumptions she lists.

What it means

For the policy. The 2x threshold was designed as an early warning, not as evidence that a catastrophe has begun. METR notes the "low, medium, high" rating reflects the risk that models directly enable catastrophic scenarios, not the risk of crossing the threshold, and that Anthropic could have made this clearer. But the report puts a number on something that was previously a policy definition.

For outside oversight. METR is unusually blunt about what it could not do. Anthropic's redactions removed detailed AI-attribution estimates, algorithmic-progress estimates, bottleneck analyses and forecasts for future models. METR says "the report does not achieve transparency about evidence", that its high-level conclusions depend significantly on redacted material, and that the public "has to rely on METR's judgment much more than would be ideal." A planned check of how compute is split between capabilities and safety was cancelled partway through.

For conflicts of interest. METR discloses that one team member had discussed a possible role with Anthropic (declined) and formerly held equity in a competitor, with their work checked by an unconflicted colleague, and that METR receives free tokens and laptops under other arrangements with Anthropic. Its contract left METR with sole discretion over methods and conclusions, while Anthropic could redact non-public information.

For readers. The report finds no convincing bottleneck in algorithms, training compute, experiment compute or data over the next 12 months, though METR says its own view changed several times. It also lists reasons the true figure could be higher (possible hyperbolic growth, benchmarks understating real-world impact) or lower (non-AI factors, gains concentrated in easily verified tasks, an unusually low baseline before late 2025).

What we don't know yet

• How much of the redacted evidence supports the 2x conclusion and in which direction.
• Whether Anthropic's own estimate will move; Anthropic has said it will publish further reports under its policy.
• Which future model the S-1 warning refers to, and what Anthropic would do on crossing the threshold. The coverage we read says the filing does not say.
• How the picture looks at other labs. METR's assessment covers only Anthropic.
• Whether the AECI, an internal variant of the Epoch Capabilities Index, is a good proxy for useful capability.

We did not open Anthropic's Risk Report itself; the description of its conclusions comes from METR's review.

What to watch next

Watch for Anthropic's next report under its policy and for any statement on whether it considers the 2x threshold crossed. Watch for similar high-access assessments of other labs, and for METR's own follow-up. Anthropic's IPO process is the other clock: the amended S-1 puts the risk in a document investors read.

Sources

• METR: Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026) (primary)
• METR: Assessment of AI R&D acceleration at Anthropic (primary)
• METR blog index
• Planned Obsolescence, Ajeya Cotra: AI is dramatically accelerating AI research right now
• AI Weekly: Anthropic amended S-1/A warns Claude may "imminently meet" its threshold