跳到正文
原文
The Decoder:AI News· Maximilian Schreiner·· 10 天前AI 评分60

METR 审计称 Anthropic 可能已在 7 月触及自家 AI 加速预警阈值

METR finds Anthropic may have crossed its own AI acceleration warning threshold back in July

文章要点
  • 1METR 审计称 Anthropic 模型测试成绩提速约 1.9 倍,从每年 15 个 AECI 点升至 29 个
  • 2METR 估计约 80% 提速来自 AI 用于其自身研究,员工非正式自评效率提升最高达 12 倍
  • 3METR 称 7 月 15 日或已触及 Anthropic 自设加速预警阈值,公开报告低于 2 倍的表述被指低估
  • 4Anthropic 拥有非公开信息单方删改权并大量使用,METR 评为最有信息量的多数证据被删除
决策要点

外部审计把 Anthropic 的进展加速量化到可能触及自家预警阈值的程度,同时也暴露出受审方删改证据带来的透明度局限。

  • 影响:前沿模型能力节奏可能在未来一年继续加快,选型与试点计划需按更快的变化频率设定假设;供应商公开安全报告可能低估指标与不确定性,风险评估不能只依据公开发布;提速主要由实验室用 AI 加速自身研究驱动,能力迭代存在自我强化的基础。
  • 参考:可考虑在模型采购评估中要求关键能力与安全声明附第三方审计或独立基准佐证;值得按半年一次的频率复核选型假设,并测算自家业务对能力快速迭代的敏感度。
  • 依据:提速 1.9 倍是 METR 基于未公开基准与约 15 次访谈的估计,区间 1.3 至 3,纳入多家厂商后行业口径为 1.4;员工 12 倍提升为非正式自评,METR 认为证据很不可靠,且最有信息量证据经删改。
正文

That's what METR set out to verify. The nonprofit studies, among other things, whether AI developers could lose control of their systems. From August 12 to October 29, 2026, a team of 2.5 full-time staffers received what they describe as unprecedented access to benchmark results from unreleased models, training metrics, employee surveys, and internal analyses, along with roughly 15 interviews with Anthropic researchers. METR took no money from Anthropic but worked on Anthropic devices for security reasons.

Anthropic CEO Dario Amodei had recently committed to bringing external auditors into the company. In October, Anthropic then announced a collaboration with METR involving "employee-like access". The current audit is a separate project with a narrower scope, according to METR.

Two years of progress packed into one

Since October 2025, Anthropic's model test scores have been climbing about 1.9 times faster than they did between March 2024 and October 2025, according to METR. The measurement is based on the AECI, a composite metric that bundles many benchmarks together. Anthropic achieved in one year what previously took two, roughly comparable to the jump from Claude 2.1 to Sonnet 4.5. The analysis also included three unreleased models that finished training between July and October 2026. Before October 2025, Anthropic's models gained about 15 AECI points per year; since then, the rate has hit 29.

Before October 2025, Anthropic's models gained about 15 AECI points per year. Since then, the rate has hit 29. Unreleased models are marked in orange. | Image: METR

METR is confident the acceleration isn't driven by faster growth in training compute. Instead, the auditors estimate that about 80 percent of the current speedup comes from AI being used in Anthropic's own research. One employee said in the interviews: "Now only bottlenecked on ideas & compute, no longer bottlenecked on code."

Others reported that experiments that used to take one to two days to set up are now "~free." In informal Slack polls from October 2026, employees said on average that the latest internal model made them roughly twelve times more productive. METR treats these self-assessments as "very loose evidence," calling them unrepresentative and likely to overstate the actual gains.

Self-reported productivity gains rose from 1.2x with Claude 3.7 Sonnet to 12x with internal Model 4. | Image: METR[

AI isn't doing fully autonomous research yet. The acceleration comes mainly from AI acting as an assistant, but there are "signs of life" that independent contributions could play a bigger role soon. METR found no bottlenecks that would slow the pace over the next twelve months, which means there's a "significant risk" that the acceleration will keep growing over the coming year.

Big uncertainty and dissenting voices

METR also stresses how imprecise the results are. The true acceleration factor could be as low as 1.3 or as high as 3. When models from OpenAI, Google, and xAI are factored in, the industry-wide number drops to just 1.4, possibly because Anthropic lagged behind competitors in 2025 and a catch-up effect is skewing the figures.

Views inside Anthropic aren't uniform either. One researcher pegs AI's share of the productivity gains at just 15 to 20 percent and distrusts colleagues' self-reports. Another considers new hires the single most important factor of recent months.

Redacted evidence weakens the audit

Anthropic had unilateral redaction rights over all non-public information and used them heavily. According to METR, the majority of the evidence the auditors rated as most informative was stripped out, including forecasts for future models, breakdowns by training steps, and the bottleneck analysis. Anthropic describes the redactions as "aimed at removing confidential information that could accelerate other AI developers."

The result is that the public has to rely on METR's judgment "much more than would be ideal," the organization writes, and any talk of transparency around the evidence is off the table. One graph in METR's report accordingly features the words "FAKE DATA" in large letters.

Anthropic pulled the plug on a compute allocation audit

In a companion review, METR also examines Anthropic's risk report from August 2026. In that report, the company publicly stated the acceleration was "less than a factor of 2." The internal, unredacted version implies a value of 1.97, though, and Anthropic itself calls it "plausible" that the threshold has been crossed. METR considers it possible that the warning level was already reached on July 15 and accuses the public report of understating both the acceleration and the uncertainty around it.

METR currently shares Anthropic's assessment that the acute catastrophic risk is "low," but the organization can't rule out another sharp acceleration over the next six months, which could quickly push the risk rating higher.

METR was originally supposed to also verify how Anthropic splits its compute between capability research and safety research. Anthropic canceled that part of the project midway through, saying it wanted to focus on risk assessment first and revisit the compute audit later.

Daniel Kokotajlo, the researcher behind the "AI 2027" scenario, responded sharply on X, arguing that auditing the compute allocation is the foundation for making commitments to slow down verifiable in the first place.

来源:The Decoder:AI News · the-decoder.com