Skip to content
AI Tracker

Agents & METR

How long a task AI can complete on its own, measured in human working time.

27 entries · 4 charts1 new · 7 updated Follow (RSS)

Progress over time

All metrics →

ARC-AGI-3 games (standard harness)

81%

nowClaude Fable 5.2Nov 2026

  1. 6 mo0.5%Best frontier AI at launch+81 pts
  2. 1 yrtest didn’t exist yet
  3. 2 yrtest didn’t exist yet
  4. 3 yrtest didn’t exist yet

ARC-AGI-3 games (lab harness)

100%

nowGPT-6 Astra + OpenAI adapterSept 2026

  1. 6 mono result yet
  2. 1 yrtest didn’t exist yet
  3. 2 yrtest didn’t exist yet
  4. 3 yrtest didn’t exist yet

ARC-AGI-2 puzzles (with harness)

No new resultslast data Feb 2026
73%

9 mo agoGPT-5.2 + refinement (johan_land)Feb 2026

  1. 6 mosame record as now
  2. 1 yr29%Grok 4 + refinement (j_berman)+44 pts
  3. 2 yrtest didn’t exist yet
  4. 3 yrtest didn’t exist yet

Charts

AI 2027 scenario vs reality

How fast each measure moved compared with the AI 2027 scenario, as graded by its authors. 1.0 = exactly on the scenario's pace.

What this measures: AI 2027 is a detailed 2025 forecast in which AI that out-codes the best humans arrives inside a lab by March 2027. A bar longer than the dashed line means reality is moving faster than the scenario on that measure.

Updated
Global AI data-centre spending~$850B/yr vs ~$510B predicted (Aug 2026)
1.76×
Leading-lab revenueAnthropic $47B annualised (May 2026)
1.43×
AI chips in the world~24.5M H100-equivalents vs ~22M predicted
1.14×
Length of tasks AI can do (METR)Claude Mythos Preview, Apr 2026
1.05×
Using a computer (OSWorld)60.8% vs 65% predicted, Aug 2025
0.84×
Top lab valuation$852B vs $1.3T predicted (Mar 2026)
0.7×
Largest training runauthors' estimate
0.58×
AI speeding up AI researchauthors' Aug 2026 estimate; METR's Nov 2026 audit points higher
0.31×
Coding benchmark (SWE-bench)74.5% vs 85% predicted for mid-2025
0.18×
Public concern about AIGallup, 4-month average to Jul 2026
0.1×

These grades stop in August 2026. Since then, METR's audit (5 November) found AI roughly doubling Anthropic's research pace, a level the scenario reaches around March 2027, so the “AI speeding up AI research” bar is out of date. GPT-6 Astra and Claude Fable 5.2 are not graded yet.

Paces are the AI Futures Project's own grading (Feb and Aug 2026) and are sensitive to how each metric is measured. They are the scenario authors' estimates, not independent ones.

Source: AI Futures ProjectData as of 10 Nov 2026
Show data table and sources
ItemValueNote
Global AI data-centre spending1.76×~$850B/yr vs ~$510B predicted (Aug 2026)
Leading-lab revenue1.43×Anthropic $47B annualised (May 2026)
AI chips in the world1.14×~24.5M H100-equivalents vs ~22M predicted
Length of tasks AI can do (METR)1.05×Claude Mythos Preview, Apr 2026
Using a computer (OSWorld)0.84×60.8% vs 65% predicted, Aug 2025
Top lab valuation0.7×$852B vs $1.3T predicted (Mar 2026)
Largest training run0.58×authors' estimate
AI speeding up AI research0.31×authors' Aug 2026 estimate; METR's Nov 2026 audit points higher
Coding benchmark (SWE-bench)0.18×74.5% vs 85% predicted for mid-2025
Public concern about AI0.1×Gallup, 4-month average to Jul 2026

ARC-AGI-3: interactive puzzle games

Best ARC Prize score on the semi-private set. Standard harness vs a lab's own harness.

What this measures: Hundreds of small turn-based games with no instructions: the AI has to explore, work out the rules and win in as few moves as possible. Every game was beaten by human testers; 100% means matching a typical first-time human player's efficiency on every game.

Updated
  • ARC's standard harness
  • Lab's own harness (provider adapter)
  • Reference level
0%50%100%Typical first-time human player (100%)

The harness matters on interactive tasks: every model tested both ways scored far higher when it could keep its own hidden reasoning between moves (e.g. GPT-6 Astra 62.7% → 99.9%, Claude Fable 5.2 81.2% → 97.7%). Dates are model release dates. ARC-AGI-3 costs are per run, not per task. ARC Prize 2026 competition results are due on 4 December 2026.

Sources: ARC Prize, ARC Prize FoundationData as of 10 Nov 2026
Show data table and sources
SeriesDateModelValue
ARC's standard harness25 Mar 2026Best frontier AI at launch0.5%
ARC's standard harness1 Jun 2026Claude Opus 4.8 (high)1.5%
ARC's standard harness9 Jul 2026GPT-5.6 Sol (max)7.8%
ARC's standard harness24 Jul 2026Claude Opus 5 (high)30%
ARC's standard harness2 Sept 2026GPT-6 Astra (max)63%
ARC's standard harness2 Sept 2026Gemini 3.8 Flash10%
ARC's standard harness28 Sept 2026Claude Sonnet 5.524%
ARC's standard harness29 Sept 2026GPT-6.1 Sol53%
ARC's standard harness30 Sept 2026Gemini 4 Argon50%
ARC's standard harness9 Nov 2026Claude Fable 5.281%
Lab's own harness (provider adapter)2 Sept 2026Gemini 3.8 Flash + adapter35%
Lab's own harness (provider adapter)3 Sept 2026GPT-6 Astra + OpenAI adapter100%
Lab's own harness (provider adapter)28 Sept 2026Claude Sonnet 5.5 + adapter72%
Lab's own harness (provider adapter)29 Sept 2026GPT-6.1 Sol + adapter96%
Lab's own harness (provider adapter)30 Sept 2026Gemini 4 Argon + adapter98%
Lab's own harness (provider adapter)9 Nov 2026Claude Fable 5.2 + adapter98%
ReferenceTypical first-time human player (100%)100%

ARC-AGI-2 puzzles: with and without a harness

Best ARC Prize-verified score on the semi-private set, by model release date.

What this measures: Visual logic puzzles that every task was solved by at least two people, but that AI found almost impossible in early 2025. A "harness" is extra code around the model that lets it try, check and refine answers.

  • Model alone (ARC's standard setup)
  • With a third-party refinement harness
  • Reference level
0%50%100%20252026Human panel (100%)Average test-taker (60%)

Model alone (ARC's standard setup), Saturated: The best model scores 95% (September 2026), and ARC Prize says ARC-AGI-1 and -2 are being overfit. Its focus has moved to the interactive ARC-AGI-3. Now watch: ARC-AGI-3.

With a third-party refinement harness, No new results · last data Feb 2026: No new verified harness result since February 2026: plain models overtook the harness systems (GPT-6 Astra scored 95% at $1.12 per task, versus 72.9% at $38.99 for the best harness), so there has been little reason to submit new ones.

Harness systems led in 2025, then plain models overtook them in 2026. The compute-capped ARC Prize 2025 Kaggle competition was won by NVARC with 24.0% at about $0.20 per task. ARC Prize has said ARC-AGI-1 and -2 are being overfit, intentionally or not.

Source: ARC PrizeData as of 9 Nov 2026
Show data table and sources
SeriesDateLabelValue
Model alone (ARC's standard setup)20 Dec 2024o3-preview (low)4%
Model alone (ARC's standard setup)16 Apr 2025o3 (high)6.5%
Model alone (ARC's standard setup)14 May 2025Claude Opus 48.6%
Model alone (ARC's standard setup)9 Jul 2025Grok 4 (thinking)16%
Model alone (ARC's standard setup)6 Oct 2025GPT-5 Pro18%
Model alone (ARC's standard setup)18 Nov 2025Gemini 3 Pro31%
Model alone (ARC's standard setup)24 Nov 2025Claude Opus 4.538%
Model alone (ARC's standard setup)11 Dec 2025GPT-5.2 Pro (high)54%
Model alone (ARC's standard setup)5 Feb 2026Claude Opus 4.6 (high)69%
Model alone (ARC's standard setup)12 Feb 2026Gemini 3 Deep Think85%
Model alone (ARC's standard setup)22 Apr 2026GPT-5.5 (xhigh)85%
Model alone (ARC's standard setup)9 Jun 2026Claude Fable 5 (max)89%
Model alone (ARC's standard setup)9 Jul 2026GPT-5.6 Sol (max)93%
Model alone (ARC's standard setup)2 Sept 2026GPT-6 Astra (max)95%
With a third-party refinement harness7 Aug 2025Grok 4 + refinement (j_berman)29%
With a third-party refinement harness4 Dec 2025Gemini 3 Pro + Poetiq54%
With a third-party refinement harness3 Feb 2026GPT-5.2 + refinement (johan_land)73%
ReferenceHuman panel (100%)100%
ReferenceAverage test-taker (60%)60%

Length of tasks AI agents can complete on their own

METR time horizon: the length of software tasks (measured in skilled-human time) an AI agent completes with 50% or 80% success. Log scale.

What this measures: How big a job an AI agent can finish on its own. A task's "length" is how long it takes a skilled human; the chart shows the longest tasks an AI completes half the time (blue) or reliably (orange). Tasks are software, machine-learning and cybersecurity work.

  • 50% success
  • 80% success
  • Exponential trend
1 s10 s1 min10 min1 h8 h40 h201920202021202220232024202520262027

Dashed line: exponential fit since 1 Jan 2023, doubling roughly every 4.0 months.

50% success, No longer updated · last data Apr 2026: METR stopped updating this measure on 8 September 2026. The newest models finish almost every task in its test suite, and METR says results above about 16 hours are unreliable, so its last data point is Claude Mythos Preview (April 2026), and no GPT-6, Claude 5.x or Gemini 4 model has a score. METR has moved to measuring how much AI speeds up AI research inside labs. Now watch: METR's AI R&D acceleration audits (e.g. Anthropic, Nov 2026) and Epoch AI/METR's MirrorCode weeks-long coding tasks.

80% success, No longer updated · last data Apr 2026: METR stopped updating this measure on 8 September 2026. The newest models finish almost every task in its test suite, and METR says results above about 16 hours are unreliable, so its last data point is Claude Mythos Preview (April 2026), and no GPT-6, Claude 5.x or Gemini 4 model has a score. METR has moved to measuring how much AI speeds up AI research inside labs. Now watch: METR's AI R&D acceleration audits (e.g. Anthropic, Nov 2026) and Epoch AI/METR's MirrorCode weeks-long coding tasks.

METR Time Horizon 1.1 values (after METR's March 2026 correction), plotted at model release dates. Tasks are well-specified software, ML and cybersecurity tasks. Confidence intervals are wide (e.g. Claude Opus 4.6: 317–3,634 min). METR stopped updating this dataset in September 2026 and says measurements above ~16 hours are unreliable with its current tasks, so models released after spring 2026 are not shown.

Source: METRData as of 9 Nov 2026
Show data table and sources
SeriesDateModelValue
50% success14 Feb 2019GPT-26 s
50% success28 May 2020davinci-002 (GPT-3)6 s
50% success15 Mar 2022GPT-3.5 Turbo Instruct36 s
50% success14 Mar 2023GPT-44 min
50% success6 Nov 2023GPT-4 11064 min
50% success4 Mar 2024Claude 3 Opus4 min
50% success9 Apr 2024GPT-4 Turbo3.7 min
50% success13 May 2024GPT-4o7 min
50% success20 Jun 2024Claude 3.5 Sonnet (Jun)11 min
50% success12 Sept 2024o1-preview20 min
50% success22 Oct 2024Claude 3.5 Sonnet (Oct)21 min
50% success5 Dec 2024o139 min
50% success24 Feb 2025Claude 3.7 Sonnet1 h
50% success16 Apr 2025o32 h
50% success22 May 2025Claude Opus 41.7 h
50% success5 Aug 2025Claude Opus 4.11.7 h
50% success7 Aug 2025GPT-53.4 h
50% success18 Nov 2025Gemini 3 Pro3.7 h
50% success19 Nov 2025GPT-5.1-Codex-Max3.7 h
50% success24 Nov 2025Claude Opus 4.54.9 h
50% success11 Dec 2025GPT-5.25.9 h
50% success5 Feb 2026Claude Opus 4.612 h
50% success5 Feb 2026GPT-5.3-Codex5.8 h
50% success19 Feb 2026Gemini 3.1 Pro6.4 h
50% success5 Mar 2026GPT-5.45.7 h
50% success7 Apr 2026Claude Mythos Preview (limited access)17 h
80% success15 Mar 2022GPT-3.5 Turbo Instruct18 s
80% success14 Mar 2023GPT-454 s
80% success6 Nov 2023GPT-4 110648 s
80% success13 May 2024GPT-4o1.3 min
80% success20 Jun 2024Claude 3.5 Sonnet (Jun)1.7 min
80% success12 Sept 2024o1-preview4.4 min
80% success5 Dec 2024o17.1 min
80% success24 Feb 2025Claude 3.7 Sonnet12 min
80% success16 Apr 2025o330 min
80% success7 Aug 2025GPT-538 min
80% success18 Nov 2025Gemini 3 Pro54 min
80% success11 Dec 2025GPT-5.21.1 h
80% success5 Feb 2026Claude Opus 4.61.2 h
80% success19 Feb 2026Gemini 3.1 Pro1.5 h
80% success7 Apr 2026Claude Mythos Preview (limited access)3.1 h

Entries

Myths

Myth check: “AI progress has hit a wall”

Verdict: False

AI kept improving fast through 2026, from 4-minute tasks in 2023 to 12-hour ones. Just making models bigger did slow, and real use lags.

Myth check5 sources