Benchmarks

• GPQA Diamond
• FrontierMath Tiers 1–3
• Humanity's Last Exam
• ARC-AGI-2
• ARC-AGI-3

GPQA and FrontierMath points are Epoch AI's independent runs (FrontierMath on the corrected 2026 set, re-run on older models); HLE is Scale AI's no-tools leaderboard; ARC points are ARC Prize-verified with the standard setup. Fast saturation partly reflects benchmark flaws and contamination, not only skill.

• Version 4.0 (January to June 2026)
• Versions 4.1 to 4.3 (June to October 2026)
• Version 5 (current)

* The index is rebuilt every few months with harder tests, so a score only means something next to other scores from the same version. Claude Fable 5.1 scored 66 on 1 September 2026 and 53 six days later, after two updates. The current version is 5.0.1 (3 November 2026).

Version 4.0 (January to June 2026), Replaced by a newer test · last data Jun 2026: Replaced by version 4.1 on 15 June 2026, which swapped several tests. Claude Fable 5 went from 64.9 to 60 overnight. Now watch: version 5.

Versions 4.1 to 4.3 (June to October 2026), Replaced by a newer test: Four updates in four months added harder, partly private tests. The top score fell from 66 to 53 between 1 and 7 September 2026. Now watch: version 5.

Version 4.0 scores are Artificial Analysis's re-scoring of older models, read from its January 2026 leaderboard and later articles. Version 5 scores were read on 10 November 2026; models released before about July 2026 have no version 5 score. For comparing across time, Artificial Analysis publishes a separate Progress Index, on which GPT-4 scores 140 and Claude Fable 5.2 scores 358. Versions 1 to 3 (2024–2025) are left out.

Labs report higher scores with tools (e.g. 64.7% for Claude Mythos Preview, self-reported). Those are not comparable with these no-tools runs. Independent reviews estimate 18–29% of the biology/chemistry answers conflict with the literature, so scores above ~70–80% may reflect label noise rather than skill.

• ARC's standard harness
• Lab's own harness (provider adapter)
• Reference level

The harness matters on interactive tasks: every model tested both ways scored far higher when it could keep its own hidden reasoning between moves (e.g. GPT-6 Astra 62.7% → 99.9%, Claude Fable 5.2 81.2% → 97.7%). Dates are model release dates. ARC-AGI-3 costs are per run, not per task. ARC Prize 2026 competition results are due on 4 December 2026.

Plotted at model release dates (the benchmark launched 1 September 2026, so earlier models were evaluated retroactively). Outside the official one-attempt rules, 17 of the 68 problems have now been solved by some means, including 12 credited to OpenAI's 6 October research release. Contamination risk grows as solutions are published.

Dates are ARC Prize's model release dates. Costs depend on ARC Prize's pricing assumptions, which it has revised (o3-preview went from $26 to $200 per task). The tested o3-preview was trained on part of the public training set and differs from the released o3.

• Best AI score (Epoch AI runs)
• Reference level

Saturated: Top scores (about 96%) are near the test's ceiling, where errors in the answer key cap what's achievable, so differences between today's best models are mostly noise. Now watch: Humanity's Last Exam.

Independent runs by Epoch AI, plotted at each model's release date (many runs happened later). The expert baseline comes from a study OpenAI ran with recruited PhDs; the original paper reports 65% on the full question set. Questions have been public since 2023, so training-data contamination can't be ruled out, and scores above ~95% may be near the benchmark's error floor.

• Top model (offline test)
• Reference level

Monthly maxima, dated to the end of each month. IQ norms are designed for humans and aren't validated for AI; this is a small matrix-style quiz, not a measure of general intelligence. On the public Mensa Norway test (which may be in training data) scores are higher, about 150.

• Model alone (ARC's standard setup)
• With a third-party refinement harness
• Reference level

Model alone (ARC's standard setup), Saturated: The best model scores 95% (September 2026), and ARC Prize says ARC-AGI-1 and -2 are being overfit. Its focus has moved to the interactive ARC-AGI-3. Now watch: ARC-AGI-3.

With a third-party refinement harness, No new results · last data Feb 2026: No new verified harness result since February 2026: plain models overtook the harness systems (GPT-6 Astra scored 95% at $1.12 per task, versus 72.9% at $38.99 for the best harness), so there has been little reason to submit new ones.

Harness systems led in 2025, then plain models overtook them in 2026. The compute-capped ARC Prize 2025 Kaggle competition was won by NVARC with 24.0% at about $0.20 per task. ARC Prize has said ARC-AGI-1 and -2 are being overfit, intentionally or not.

• Independent (Epoch AI, fixed setup)
• Self-reported by labs

Independent (Epoch AI, fixed setup), Known flaws · last data Apr 2026: Epoch AI stopped adding runs after June 2026, and OpenAI stopped reporting this test in February 2026 after finding flawed tests and signs that models had memorised the fixes. Top scores were also near the ceiling. Now watch: SWE-bench Pro (Scale AI) and MirrorCode.

Self-reported by labs, Replaced by a newer test · last data Apr 2026: Labs largely stopped reporting SWE-bench Verified in 2026 after OpenAI's audit found flawed tests and memorised fixes; Anthropic's last reported score was 93.9% (April 2026). Now watch: SWE-bench Pro (Scale AI).

Epoch AI stopped adding SWE-bench runs after June 2026, and OpenAI stopped reporting the benchmark in February 2026 after finding flawed tests and signs that models had memorised fixes. Lab numbers use their own agent setups and run higher; the successor benchmark is SWE-bench Pro.

Saturated · last data Apr 2026: Top models score 100% (GPT-5.5 Pro, April 2026), so this test can no longer show progress and labs stopped reporting it. Now watch: FrontierMath and FrontierMath Erdős (open problems).

Release dates for GPT-4 (0613), Claude 2.0, Claude 3 Opus and Llama 3.1 are the models' public release dates. Each problem is worth about 2.2 points. Labs stopped reporting AIME in 2026 because top models solve it all.

• Tiers 1–3 (advanced)
• Tier 4 (research-level)

Tier 4 (research-level), Saturated: Epoch AI considers Tier 4 saturated: GPT-6 Astra solved the last problem no model had solved (September 2026), and GPT-6.1 Sol scores 100%. Now watch: FrontierMath Erdős (68 genuinely open problems).

Epoch AI re-ran older models on the corrected v2 problem set (June 2026), so early points come from those re-runs. OpenAI funded FrontierMath and has access to many problems; Epoch keeps a held-out set. Answers are checked automatically. These are not proofs of open problems.

• 50% success
• 80% success
• Exponential trend

Dashed line: exponential fit since 1 Jan 2023, doubling roughly every 4.0 months.

50% success, No longer updated · last data Apr 2026: METR stopped updating this measure on 8 September 2026. The newest models finish almost every task in its test suite, and METR says results above about 16 hours are unreliable, so its last data point is Claude Mythos Preview (April 2026), and no GPT-6, Claude 5.x or Gemini 4 model has a score. METR has moved to measuring how much AI speeds up AI research inside labs. Now watch: METR's AI R&D acceleration audits (e.g. Anthropic, Nov 2026) and Epoch AI/METR's MirrorCode weeks-long coding tasks.

80% success, No longer updated · last data Apr 2026: METR stopped updating this measure on 8 September 2026. The newest models finish almost every task in its test suite, and METR says results above about 16 hours are unreliable, so its last data point is Claude Mythos Preview (April 2026), and no GPT-6, Claude 5.x or Gemini 4 model has a score. METR has moved to measuring how much AI speeds up AI research inside labs. Now watch: METR's AI R&D acceleration audits (e.g. Anthropic, Nov 2026) and Epoch AI/METR's MirrorCode weeks-long coding tasks.

METR Time Horizon 1.1 values (after METR's March 2026 correction), plotted at model release dates. Tasks are well-specified software, ML and cybersecurity tasks. Confidence intervals are wide (e.g. Claude Opus 4.6: 317–3,634 min). METR stopped updating this dataset in September 2026 and says measurements above ~16 hours are unreliable with its current tasks, so models released after spring 2026 are not shown.