- Epoch Capabilities Index174 points
- AA Intelligence Index*64.6 points
- Remote Labor Index41%
- FrontierSWE V270%
Model tracker
Today's leading models from the big three labs, and every model generation on the same tests. Each score links to where it was measured.
The current big three
The strongest model each of Anthropic, OpenAI and Google offers today, newest first.
AnthropicLatest releaseClaude Fable 5.2 GoogleGemini 4 Carbon - Epoch Capabilities Indexnot tested
- AA Intelligence Index*60 points
- Remote Labor Indexnot tested
- FrontierSWE V261%
OpenAIGPT-6 Astra - Epoch Capabilities Index169 points
- AA Intelligence Index*56.9 points
- Remote Labor Index21%
- FrontierSWE V266%
Compare models on the same test
Pick a benchmark, then choose the models. Benchmarks that no longer tell models apart are marked.
What does it mean when a test is “saturated”?
A test is saturated when the best AI scores close to the maximum, or as high as mistakes in the test's own answer key allow. It can no longer tell newer models apart, so researchers build harder tests.Full glossary.
Tests designed to stay hard for years have often fallen in one or two. Skill isn't the only cause: some tests leak into training data or have flawed questions. Each chart says when and why a test stopped being useful, and what replaced it.
What is a “human baseline”?
The score people get on the same test, measured by the test's creators. PhD experts get about 70% of GPQA's science questions right; the average person solves about 60% of ARC-AGI-2's puzzles.
Baselines depend on who was tested (specialists or the general public), how long they had and whether they were paid to try hard. “AI beats humans” only means something when it says which humans, on which test. Many tests have no published human baseline.
Epoch Capabilities Index (many tests combined into one score)
What this measures: One number for general ability, made by combining a model's results on more than 50 tests, so models stay comparable after single tests max out. GPT-4 (2023) scored about 122 and GPT-5 (2025) 150. Scores shift by a point or so as Epoch AI adds results, and each carries a margin of error of about 2 to 3 points.
Claude Fable 5.2Nov 2026Latest release174 points
GPT-6 AstraSept 2026169 points
GPT-6.1 SolSept 2026169 points
Claude Opus 5.5Sept 2026169 points
GPT-6.1 TerraOct 2026168 points
Claude Fable 5.1Sept 2026167 points
Gemini 4 ArgonOct 2026167 points
Claude Sonnet 5.5Sept 2026166 points
Claude Opus 5Jul 2026164 points
Claude Fable 5Jun 2026164 points
GPT-5.6 SolJul 2026163 points
Kimi K3.1 ProNov 2026162 points
Muse WonderOct 2026162 points
Claude Haiku 5.5Oct 2026160 points
GPT-5.5Apr 2026160 points
Claude Opus 4.8May 2026160 points
Gemini 3.8 FlashSept 2026159 points
GPT-5.4Mar 2026158 points
Claude Opus 4.7Apr 2026158 points
Qwen3.8 Max (0902)Aug 2026157 points
DeepSeek V4.1 Flash (Oct 2026)Oct 2026156 points
Claude Opus 4.6Feb 2026156 points
Gemini 3.1 ProFeb 2026156 points
Grok 4.7Sept 2026156 points
GPT-5.2Dec 2025154 points
Gemini 3 ProNov 2025153 points
Claude Opus 4.5Nov 2025152 points
GPT-5Aug 2025150 points
o3Apr 2025147 points
Gemini 2.5 ProJun 2025145 points
Claude Opus 4May 2025143 points
o1Dec 2024141 points
GPT-4oMay 2024129 points
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Epoch AI, Epoch AI.
Artificial Analysis Intelligence Index v5* (about ten tests in one score)
What this measures: One score out of 100 from about ten tests of reasoning, coding, knowledge work and agent tasks, run by Artificial Analysis. * These are version 5.0.1 scores. The index is rebuilt every few months with harder tests, so they cannot be compared with scores quoted before 27 October 2026, and models released before about July 2026 have no version 5 score.
Claude Fable 5.2Nov 2026Latest release64.6 points
Gemini 4 CarbonOct 202660 points
Claude Opus 5.5Sept 202659.1 points
GPT-6 AstraSept 202656.9 points
GPT-6.1 SolSept 202654.6 points
Claude Sonnet 5.5Sept 202654.5 points
GPT-6.1 TerraOct 202651.3 points
Gemini 4 ArgonOct 202650.4 points
Claude Fable 5.1Sept 202650 points
Kimi K3.1 ProNov 202646.5 points
Claude Opus 5Jul 202644.2 points
Muse WonderOct 202643.6 points
Claude Haiku 5.5Oct 202636.5 points
Grok 4.7Sept 202635.5 points
Gemini 3.8 FlashSept 202630.8 points
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o3Apr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Gemini 2.5 ProJun 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
GPT-5Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Opus 4.5Nov 2025no independent score on this test
GPT-5.2Dec 2025no independent score on this test
Claude Opus 4.6Feb 2026no independent score on this test
Gemini 3.1 ProFeb 2026no independent score on this test
GPT-5.4Mar 2026no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Claude Opus 4.7Apr 2026no independent score on this test
GPT-5.5Apr 2026no independent score on this test
Claude Opus 4.8May 2026no independent score on this test
Claude Fable 5Jun 2026no independent score on this test
GPT-5.6 SolJul 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Artificial Analysis, Artificial Analysis.
Remote Labor Index (paid freelance projects)
What this measures: 230 real freelance projects, from game development and product design to data analysis and video animation, that professionals were paid to do. The score is the share an AI agent finishes to a standard a client would accept. The average project took a person 29 hours.
Claude Fable 5.2Nov 2026Latest release41%
Claude Opus 5.5Sept 202621%
GPT-6 AstraSept 202621%
Claude Fable 5.1Sept 202618%
Gemini 4 ArgonOct 202617%
Claude Fable 5Jun 202616%
Claude Opus 4.8May 20268.3%
GPT-5.5Apr 20266.3%
Gemini 3.8 FlashSept 20265.8%
Claude Opus 4.6Feb 20264.2%
Claude Opus 4.5Nov 20253.8%
GPT-5.2Dec 20252.5%
GPT-5Aug 20251.7%
Gemini 3 ProNov 20251.3%
Gemini 2.5 ProJun 20250.8%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o3Apr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3.1 ProFeb 2026no independent score on this test
GPT-5.4Mar 2026no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Claude Opus 4.7Apr 2026no independent score on this test
GPT-5.6 SolJul 2026no independent score on this test
Claude Opus 5Jul 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
Grok 4.7Sept 2026no independent score on this test
Claude Sonnet 5.5Sept 2026no independent score on this test
GPT-6.1 SolSept 2026no independent score on this test
Claude Haiku 5.5Oct 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Center for AI Safety and Scale AI, Scale AI, Mazeika et al..
FrontierSWE V2 (very long engineering jobs)
What this measures: 34 very long software engineering and research jobs, with up to 20 hours allowed for each. Every job is scored from 0 to 1 and the result is the average over five runs. The benchmark's makers, Proximal, run every model themselves at its highest reasoning setting.
Claude Fable 5.2Nov 2026Latest release70%
GPT-6 AstraSept 202666%
Claude Opus 5.5Sept 202662%
Claude Sonnet 5.5Sept 202662%
Gemini 4 CarbonOct 202661%
Claude Fable 5.1Sept 202656%
Gemini 4 ArgonOct 202655%
Claude Opus 5Jul 202652%
Claude Fable 5Jun 202647%
Claude Haiku 5.5Oct 202644%
GPT-5.6 SolJul 202632%
Grok 4.7Sept 202630%
Gemini 3.8 FlashSept 202620%
Qwen3.8 Max (0902)Aug 202618%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o3Apr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Gemini 2.5 ProJun 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
GPT-5Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Opus 4.5Nov 2025no independent score on this test
GPT-5.2Dec 2025no independent score on this test
Claude Opus 4.6Feb 2026no independent score on this test
Gemini 3.1 ProFeb 2026no independent score on this test
GPT-5.4Mar 2026no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Claude Opus 4.7Apr 2026no independent score on this test
GPT-5.5Apr 2026no independent score on this test
Claude Opus 4.8May 2026no independent score on this test
GPT-6.1 SolSept 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Proximal, Proximal.
Terminal-Bench 4.1 (long command-line tasks)
What this measures: Hard, long-running jobs done in a real computer terminal: software, data, science and system administration. The score is the share of attempts finished correctly, over 330 attempts per model. Each model runs inside its maker's own agent software, and the site does not say who ran the attempts.
Claude Opus 5.5Sept 202666%
Claude Sonnet 5.5Sept 202664%
GPT-6.1 SolSept 202661%
GPT-6 AstraSept 202660%
Claude Fable 5.1Sept 202658%
Claude Opus 5Jul 202653%
Claude Fable 5Jun 202645%
Grok 4.7Sept 202639%
GPT-5.6 SolJul 202638%
Qwen3.8 Max (0902)Aug 202627%
Claude Opus 4.8May 202624%
Gemini 3.8 FlashSept 202618%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o3Apr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Gemini 2.5 ProJun 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
GPT-5Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Opus 4.5Nov 2025no independent score on this test
GPT-5.2Dec 2025no independent score on this test
Claude Opus 4.6Feb 2026no independent score on this test
Gemini 3.1 ProFeb 2026no independent score on this test
GPT-5.4Mar 2026no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Claude Opus 4.7Apr 2026no independent score on this test
GPT-5.5Apr 2026no independent score on this test
Claude Haiku 5.5Oct 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
Gemini 4 ArgonOct 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
Claude Fable 5.2Nov 2026Latest releaseno independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Terminal-Bench.
MirrorCode (rebuilding whole programs)
What this measures: The AI must rebuild a whole command-line program from scratch, without seeing its code, so that it passes every test of the original. Tasks are estimated at days to weeks of human work. With only 30 tasks, each score carries a margin of error of 8 to 12 points. Built by Epoch AI with METR.
Claude Opus 5.5Sept 202679%
Claude Fable 5.1Sept 202673%
Claude Fable 5Jun 202664%
Gemini 4 ArgonOct 202659%
GPT-6 AstraSept 202647%
Claude Opus 4.7Apr 202631%
GPT-5.6 SolJul 202620%
GPT-5.4Mar 202616%
GPT-5.5Apr 202610%
Gemini 3.1 ProFeb 20268.9%
GPT-5Aug 20254.4%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o3Apr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Gemini 2.5 ProJun 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Opus 4.5Nov 2025no independent score on this test
GPT-5.2Dec 2025no independent score on this test
Claude Opus 4.6Feb 2026no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Claude Opus 4.8May 2026no independent score on this test
Claude Opus 5Jul 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
Gemini 3.8 FlashSept 2026no independent score on this test
Grok 4.7Sept 2026no independent score on this test
Claude Sonnet 5.5Sept 2026no independent score on this test
GPT-6.1 SolSept 2026no independent score on this test
Claude Haiku 5.5Oct 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
Claude Fable 5.2Nov 2026Latest releaseno independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Epoch AI.
APEX-Agents (professional office tasks)
What this measures: Long tasks across several applications, written by practising lawyers, consultants and investment bankers. The score is the share an AI agent gets fully right on its first attempt. Mercor runs the tests; answers are graded by an AI judge against the experts' marking schemes.
Gemini 4 CarbonOct 202684%
Gemini 4 ArgonOct 202682%
Claude Fable 5.2Nov 2026Latest release80%
Claude Sonnet 5.5Sept 202676%
Claude Opus 5.5Sept 202674%
DeepSeek V4.1 Flash (Oct 2026)Oct 202670%
Claude Fable 5.1Sept 202669%
GPT-6.1 TerraOct 202668%
Muse WonderOct 202666%
Claude Opus 5Jul 202666%
Kimi K3.1 ProNov 202666%
GPT-6 AstraSept 202665%
Gemini 3.8 FlashSept 202664%
Claude Fable 5Jun 202664%
Qwen3.8 Max (0902)Aug 202663%
GPT-6.1 SolSept 202660%
Claude Haiku 5.5Oct 202656%
GPT-5.5Apr 202655%
Grok 4.7Sept 202655%
GPT-5.4Mar 202652%
GPT-5.6 SolJul 202651%
Claude Opus 4.7Apr 202649%
Claude Opus 4.8May 202649%
Claude Opus 4.6Feb 202646%
Gemini 3.1 ProFeb 202635%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o3Apr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Gemini 2.5 ProJun 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
GPT-5Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Opus 4.5Nov 2025no independent score on this test
GPT-5.2Dec 2025no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Mercor.
Humanity's Last Exam (expert questions, no tools)
What this measures: 2,500 questions from about 1,000 experts, chosen because the best AI of early 2025 got them wrong.
GPT-6 AstraSept 202655%
Claude Fable 5.1Sept 202647%
Gemini 3.1 ProFeb 202646%
Claude Sonnet 5.5Sept 202646%
Gemini 3.8 FlashSept 202645%
Gemini 3 ProNov 202538%
GPT-5.4Mar 202636%
Claude Opus 4.7Apr 202636%
Claude Opus 4.6Feb 202634%
Claude Haiku 5.5Oct 202630%
GPT-5.2Dec 202528%
GPT-5Aug 202525%
Claude Opus 4.5Nov 202525%
Gemini 2.5 ProJun 202522%
o3Apr 202520%
o4-miniApr 202518%
Claude Opus 4.1Aug 202512%
Claude Opus 4May 202511%
Claude 3.7 SonnetFeb 20258%
o1Dec 20248%
Llama 4 MaverickApr 20255.7%
Gemini 1.5 Pro (002)May 20244.6%
Claude 3.5 Sonnet (Oct 2024)Oct 20244.1%
GPT-4oMay 20242.7%
GPT-4o (Nov 2024)Nov 20242.7%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
GPT-5.5Apr 2026no independent score on this test
Claude Opus 4.8May 2026no independent score on this test
Claude Fable 5Jun 2026no independent score on this test
GPT-5.6 SolJul 2026no independent score on this test
Claude Opus 5Jul 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
Grok 4.7Sept 2026no independent score on this test
Claude Opus 5.5Sept 2026no independent score on this test
GPT-6.1 SolSept 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
Gemini 4 ArgonOct 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
Claude Fable 5.2Nov 2026Latest releaseno independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Scale AI.
ARC-AGI-3 (puzzle games, standard harness)
What this measures: Small turn-based games with no instructions: the AI must explore, work out the rules and win in as few moves as possible. Every game was beaten by human testers; 100% means matching a typical first-time human player's efficiency.
Typical first-time human playerhuman baseline100%
Claude Fable 5.2Nov 2026Latest release81%
GPT-6 AstraSept 202663%
GPT-6.1 SolSept 202653%
Gemini 4 ArgonOct 202650%
Claude Opus 5Jul 202630%
Claude Sonnet 5.5Sept 202624%
Gemini 3.8 FlashSept 202610%
GPT-5.6 SolJul 20267.8%
Grok 4.7Sept 20261.8%
Claude Opus 4.8May 20261.5%
Claude Opus 4.6Feb 20260.5%
Claude Haiku 5.5Oct 20260.5%
GPT-5.5Apr 20260.4%
Gemini 3.1 ProFeb 20260.4%
DeepSeek V4.1 Flash (Oct 2026)Oct 20260.2%
GPT-5.4Mar 20260.2%
Claude Opus 4.7Apr 20260.2%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o3Apr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Gemini 2.5 ProJun 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
GPT-5Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Opus 4.5Nov 2025no independent score on this test
GPT-5.2Dec 2025no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Claude Fable 5Jun 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
Claude Fable 5.1Sept 2026no independent score on this test
Claude Opus 5.5Sept 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
Grey bar and dashed line: typical first-time human player (100%). Click a score to open its source. * = reported by the lab, not independently measured. Source: ARC Prize.
FrontierMath Tiers 1–3 (hard math problems)
What this measures: Unpublished problems written by mathematicians that take experts hours to days. Scores from Epoch AI's corrected 2026 problem set.
Claude Fable 5.2Nov 2026Latest release95%
GPT-6 AstraSept 202694%
GPT-6.1 SolSept 202694%
Claude Opus 5.5Sept 202691%
GPT-6.1 TerraOct 202691%
Claude Fable 5.1Sept 202690%
GPT-5.6 SolJul 202689%
Claude Sonnet 5.5Sept 202689%
Claude Fable 5Jun 202687%
Gemini 4 ArgonOct 202686%
Claude Opus 5Jul 202686%
GPT-5.5Apr 202685%
Muse WonderOct 202682%
Claude Opus 4.8May 202680%
GPT-5.4Mar 202679%
Kimi K3.1 ProNov 202678%
Claude Haiku 5.5Oct 202675%
DeepSeek V4.1 Flash (Oct 2026)Oct 202671%
Claude Opus 4.7Apr 202670%
Gemini 3.8 FlashSept 202668%
GPT-5.2Dec 202567%
Claude Opus 4.6Feb 202666%
Qwen3.8 Max (0902)Aug 202666%
Gemini 3.1 ProFeb 202660%
GPT-5Aug 202555%
Grok 4.7Sept 202653%
o4-miniApr 202536%
Claude Opus 4.5Nov 202534%
o3Apr 202533%
Gemini 2.5 ProJun 202525%
o1Dec 202415%
Claude Opus 4.1Aug 202513%
GPT-4 Turbo (Apr 2024)Apr 20240.7%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Epoch AI.
OSWorld-Verified (using a computer)
What this measures: Real desktop tasks (office apps, a browser, files) done by looking at the screen and using mouse and keyboard. People completed 72% of the original tasks.
People (original task set)human baseline72%
Claude Fable 5Jun 202686%
Claude Opus 5Jul 202683%
Claude 3.7 SonnetFeb 202536%
o3Apr 202523%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Gemini 2.5 ProJun 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
GPT-5Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Opus 4.5Nov 2025no independent score on this test
GPT-5.2Dec 2025no independent score on this test
Claude Opus 4.6Feb 2026no independent score on this test
Gemini 3.1 ProFeb 2026no independent score on this test
GPT-5.4Mar 2026no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Claude Opus 4.7Apr 2026no independent score on this test
GPT-5.5Apr 2026no independent score on this test
Claude Opus 4.8May 2026no independent score on this test
GPT-5.6 SolJul 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
Claude Fable 5.1Sept 2026no independent score on this test
Gemini 3.8 FlashSept 2026no independent score on this test
GPT-6 AstraSept 2026no independent score on this test
Grok 4.7Sept 2026no independent score on this test
Claude Opus 5.5Sept 2026no independent score on this test
Claude Sonnet 5.5Sept 2026no independent score on this test
GPT-6.1 SolSept 2026no independent score on this test
Claude Haiku 5.5Oct 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
Gemini 4 ArgonOct 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
Claude Fable 5.2Nov 2026Latest releaseno independent score on this test
Grey bar and dashed line: people (original task set) (72%). Click a score to open its source. * = reported by the lab, not independently measured. Source: XLANG Lab, HKU.
GPQA Diamond (PhD-level science questions)
What this measures: 198 hard biology, chemistry and physics questions written by PhDs. Guessing gets 25%; PhD experts in the field get about 70%.
Saturated: Top scores (about 96%) are near the test's ceiling, where errors in the answer key cap what's achievable, so differences between today's best models are mostly noise. Now watch: Humanity's Last Exam.
PhD expertshuman baseline70%
Gemini 4 ArgonOct 202696%
GPT-6 AstraSept 202696%
Claude Sonnet 5.5Sept 202696%
Gemini 3.8 FlashSept 202695%
GPT-6.1 SolSept 202695%
Gemini 3.1 ProFeb 202694%
GPT-6.1 TerraOct 202694%
Kimi K3.1 ProNov 202694%
GPT-5.5Apr 202694%
Claude Fable 5.2Nov 2026Latest release94%
Claude Opus 5Jul 202694%
GPT-5.6 SolJul 202694%
GPT-5.4Mar 202693%
DeepSeek V4.1 Flash (Oct 2026)Oct 202693%
Grok 4.7Sept 202693%
Gemini 3 ProNov 202593%
Qwen3.8 Max (0902)Aug 202692%
GPT-5.2Dec 202591%
Claude Opus 4.8May 202691%
Claude Opus 5.5Sept 202691%
Claude Opus 4.6Feb 202691%
Claude Opus 4.7Apr 202690%
Claude Haiku 5.5Oct 202690%
Grok 4Jul 202587%
GPT-5Aug 202586%
Claude Opus 4.5Nov 202586%
Claude Fable 5Jun 202686%
Gemini 2.5 ProJun 202585%
Kimi K2 ThinkingNov 202584%
o3Apr 202582%
Qwen3-235B-A22B-Thinking-2507Jul 202580%
Claude 3.7 SonnetFeb 202580%
o4-miniApr 202580%
Claude Opus 4.1Aug 202577%
o1Dec 202477%
Claude Opus 4May 202576%
DeepSeek-R1Jan 202572%
Llama 4 MaverickApr 202567%
Gemini 2.0 FlashFeb 202564%
Gemini 1.5 Pro (002)May 202457%
Claude 3.5 Sonnet (Oct 2024)Oct 202455%
GPT-4oMay 202449%
GPT-4o (Nov 2024)Nov 202448%
Claude 3 OpusMar 202447%
GPT-4 Turbo (Apr 2024)Apr 202447%
GPT-4Mar 202336%
Claude Mythos PreviewApr 2026no independent score on this test
Claude Fable 5.1Sept 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Grey bar and dashed line: phd experts (70%). Click a score to open its source. * = reported by the lab, not independently measured. Source: Epoch AI.
FrontierMath Tier 4 (research-level math problems)
What this measures: Research-level problems written by math professors and postdocs. Scores from Epoch AI's corrected 2026 problem set.
Saturated: Epoch AI considers Tier 4 saturated: GPT-6 Astra solved the last problem no model had solved (September 2026), and GPT-6.1 Sol scores 100%. Now watch: FrontierMath Erdős (68 genuinely open problems).
GPT-6.1 SolSept 2026100%
Claude Fable 5.2Nov 2026Latest release100%
GPT-6 AstraSept 202698%
Claude Opus 5.5Sept 202695%
GPT-6.1 TerraOct 202693%
Claude Fable 5Jun 202690%
Claude Fable 5.1Sept 202688%
GPT-5.6 SolJul 202683%
Claude Sonnet 5.5Sept 202680%
Gemini 4 ArgonOct 202678%
Claude Opus 5Jul 202673%
GPT-5.5Apr 202673%
Muse WonderOct 202671%
Kimi K3.1 ProNov 202666%
Claude Opus 4.8May 202656%
GPT-5.4Mar 202649%
Claude Haiku 5.5Oct 202646%
DeepSeek V4.1 Flash (Oct 2026)Oct 202646%
Qwen3.8 Max (0902)Aug 202634%
Claude Opus 4.7Apr 202632%
GPT-5.2Dec 202532%
Claude Opus 4.6Feb 202627%
Gemini 3.1 ProFeb 202627%
GPT-5Aug 202522%
Gemini 3.8 FlashSept 202622%
Grok 4.7Sept 202617%
o4-miniApr 20254.9%
Claude Opus 4.5Nov 20254.9%
Claude Opus 4.1Aug 20252.4%
Gemini 2.5 ProJun 20250%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Claude 3.7 SonnetFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o3Apr 2025no independent score on this test
Claude Opus 4May 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Gemini 3 ProNov 2025no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Epoch AI.
ARC-AGI-2 (visual puzzles, model alone)
What this measures: Visual logic puzzles every one of which was solved by at least two people. The average test-taker scores 60%.
Saturated: The best model scores 95% (September 2026), and ARC Prize says ARC-AGI-1 and -2 are being overfit. Its focus has moved to the interactive ARC-AGI-3. Now watch: ARC-AGI-3.
Average personhuman baseline60%
GPT-6 AstraSept 202695%
GPT-6.1 SolSept 202694%
Claude Fable 5.2Nov 2026Latest release94%
Claude Opus 5.5Sept 202693%
GPT-5.6 SolJul 202693%
Gemini 4 ArgonOct 202691%
Claude Opus 5Jul 202690%
Claude Fable 5.1Sept 202690%
Claude Fable 5Jun 202689%
Gemini 3.8 FlashSept 202689%
GPT-5.5Apr 202685%
Claude Sonnet 5.5Sept 202684%
DeepSeek V4.1 Flash (Oct 2026)Oct 202678%
Gemini 3.1 ProFeb 202677%
Claude Opus 4.7Apr 202676%
GPT-5.4Mar 202674%
Claude Opus 4.8May 202672%
Claude Opus 4.6Feb 202669%
Grok 4.7Sept 202661%
Claude Haiku 5.5Oct 202661%
GPT-5.2Dec 202553%
Claude Opus 4.5Nov 202538%
Gemini 3 ProNov 202531%
Grok 4Jul 202516%
GPT-5Aug 20259.9%
Claude Opus 4May 20258.6%
o3Apr 20256.5%
o4-miniApr 20256.1%
Gemini 2.5 ProJun 20254.9%
DeepSeek-R1Jan 20251.3%
Claude 3.7 SonnetFeb 20250.9%
GPT-4o (Nov 2024)Nov 20240%
Llama 4 MaverickApr 20250%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
o1Dec 2024no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Claude Opus 4.1Aug 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
Grey bar and dashed line: average person (60%). Click a score to open its source. * = reported by the lab, not independently measured. Source: ARC Prize.
OTIS Mock AIME (competition math)
What this measures: Problems at the level of the American Invitational Mathematics Examination, the qualifier for the US Math Olympiad team.
Saturated: Top models score 100% (GPT-5.5 Pro, April 2026), so this test can no longer show progress and labs stopped reporting it. Now watch: FrontierMath and FrontierMath Erdős (open problems).
GPT-5.5Apr 2026100%
Claude Fable 5Jun 2026100%
GPT-5.6 SolJul 2026100%
Qwen3.8 Max (0902)Aug 2026100%
Claude Fable 5.1Sept 2026100%
GPT-6 AstraSept 2026100%
Claude Opus 5.5Sept 2026100%
Claude Sonnet 5.5Sept 2026100%
GPT-6.1 SolSept 2026100%
Gemini 4 ArgonOct 2026100%
GPT-6.1 TerraOct 2026100%
Kimi K3.1 ProNov 2026100%
Claude Fable 5.2Nov 2026Latest release100%
Muse WonderOct 202699%
DeepSeek V4.1 Flash (Oct 2026)Oct 202699%
Claude Opus 5Jul 202699%
Gemini 3.8 FlashSept 202699%
Claude Haiku 5.5Oct 202699%
Claude Opus 4.8May 202698%
Grok 4.7Sept 202698%
Claude Opus 4.7Apr 202698%
GPT-5.4Mar 202698%
GPT-5.2Dec 202596%
Gemini 3.1 ProFeb 202696%
Claude Opus 4.6Feb 202694%
GPT-5Aug 202591%
Gemini 3 ProNov 202591%
Qwen3-235B-A22B-Thinking-2507Jul 202587%
Claude Opus 4.5Nov 202586%
Gemini 2.5 ProJun 202585%
o3Apr 202584%
Grok 4Jul 202584%
Kimi K2 ThinkingNov 202583%
o4-miniApr 202582%
o1Dec 202473%
Claude Opus 4.1Aug 202569%
Claude Opus 4May 202564%
Claude 3.7 SonnetFeb 202558%
DeepSeek-R1Jan 202553%
Gemini 2.0 FlashFeb 202531%
Gemini 1.5 Pro (002)May 202423%
Llama 4 MaverickApr 202521%
Claude 3.5 Sonnet (Oct 2024)Oct 20248.5%
GPT-4 Turbo (Apr 2024)Apr 20246.7%
GPT-4oMay 20246.3%
GPT-4o (Nov 2024)Nov 20246.3%
Claude 3 OpusMar 20244.7%
GPT-4Mar 20230.6%
Claude Mythos PreviewApr 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Epoch AI.
SWE-bench Verified (fixing real software bugs)
What this measures: The AI gets a real bug report from a popular open-source project and must change the code so the project's own tests pass.
Known flaws: Epoch AI stopped adding runs after June 2026, and OpenAI stopped reporting this test in February 2026 after finding flawed tests and signs that models had memorised the fixes. Top scores were also near the ceiling. Now watch: FrontierSWE V2, Terminal-Bench 4.1 and MirrorCode.
Claude Opus 4.7Apr 202683%
GPT-5.5Apr 202681%
Claude Opus 4.6Feb 202679%
GPT-5.4Mar 202677%
Claude Opus 4.5Nov 202577%
Gemini 3.1 ProFeb 202676%
GPT-5.2Dec 202574%
GPT-5Aug 202574%
Claude Opus 4.1Aug 202573%
Gemini 3 ProNov 202573%
Claude Opus 4May 202571%
o3Apr 202562%
Claude 3.7 SonnetFeb 202561%
Gemini 2.5 ProJun 202558%
GPT-4o (Nov 2024)Nov 202431%
GPT-4Mar 2023no independent score on this test
Claude 3 OpusMar 2024no independent score on this test
GPT-4 Turbo (Apr 2024)Apr 2024no independent score on this test
GPT-4oMay 2024no independent score on this test
Gemini 1.5 Pro (002)May 2024no independent score on this test
Claude 3.5 Sonnet (Oct 2024)Oct 2024no independent score on this test
o1Dec 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Claude Mythos PreviewApr 2026no independent score on this test
Claude Opus 4.8May 2026no independent score on this test
Claude Fable 5Jun 2026no independent score on this test
GPT-5.6 SolJul 2026no independent score on this test
Claude Opus 5Jul 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
Claude Fable 5.1Sept 2026no independent score on this test
Gemini 3.8 FlashSept 2026no independent score on this test
GPT-6 AstraSept 2026no independent score on this test
Grok 4.7Sept 2026no independent score on this test
Claude Opus 5.5Sept 2026no independent score on this test
Claude Sonnet 5.5Sept 2026no independent score on this test
GPT-6.1 SolSept 2026no independent score on this test
Claude Haiku 5.5Oct 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
Gemini 4 ArgonOct 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
Claude Fable 5.2Nov 2026Latest releaseno independent score on this test
No published human baseline for this test. Click a score to open its source. * = reported by the lab, not independently measured. Source: Epoch AI.
METR time horizon (task length an AI agent finishes half the time)
What this measures: The length of software tasks (measured in how long a skilled human takes) that the AI completes on its own half the time.
No longer updated: METR stopped updating this measure on 8 September 2026. The newest models finish almost every task in its test suite, and METR says results above about 16 hours are unreliable, so its last data point is Claude Mythos Preview (April 2026), and no GPT-6, Claude 5.x or Gemini 4 model has a score. METR has moved to measuring how much AI speeds up AI research inside labs. Now watch: MirrorCode and the Remote Labor Index, plus METR's AI R&D acceleration audits.
Claude Mythos PreviewApr 202617 h
Claude Opus 4.6Feb 202612 h
Gemini 3.1 ProFeb 20266.4 h
GPT-5.2Dec 20255.9 h
GPT-5.4Mar 20265.7 h
Claude Opus 4.5Nov 20254.9 h
Gemini 3 ProNov 20253.7 h
GPT-5Aug 20253.4 h
o3Apr 20252 h
Claude Opus 4.1Aug 20251.7 h
Claude Opus 4May 20251.7 h
Claude 3.7 SonnetFeb 20251 h
o1Dec 202439 min
Claude 3.5 Sonnet (Oct 2024)Oct 202421 min
GPT-4oMay 20247 min
GPT-4Mar 20234 min
Claude 3 OpusMar 20244 min
GPT-4 Turbo (Apr 2024)Apr 20243.7 min
Gemini 1.5 Pro (002)May 2024no independent score on this test
GPT-4o (Nov 2024)Nov 2024no independent score on this test
DeepSeek-R1Jan 2025no independent score on this test
Gemini 2.0 FlashFeb 2025no independent score on this test
Llama 4 MaverickApr 2025no independent score on this test
o4-miniApr 2025no independent score on this test
Gemini 2.5 ProJun 2025no independent score on this test
Grok 4Jul 2025no independent score on this test
Qwen3-235B-A22B-Thinking-2507Jul 2025no independent score on this test
Kimi K2 ThinkingNov 2025no independent score on this test
Claude Opus 4.7Apr 2026no independent score on this test
GPT-5.5Apr 2026no independent score on this test
Claude Opus 4.8May 2026no independent score on this test
Claude Fable 5Jun 2026no independent score on this test
GPT-5.6 SolJul 2026no independent score on this test
Claude Opus 5Jul 2026no independent score on this test
Qwen3.8 Max (0902)Aug 2026no independent score on this test
Claude Fable 5.1Sept 2026no independent score on this test
Gemini 3.8 FlashSept 2026no independent score on this test
GPT-6 AstraSept 2026no independent score on this test
Grok 4.7Sept 2026no independent score on this test
Claude Opus 5.5Sept 2026no independent score on this test
Claude Sonnet 5.5Sept 2026no independent score on this test
GPT-6.1 SolSept 2026no independent score on this test
Claude Haiku 5.5Oct 2026no independent score on this test
Muse WonderOct 2026no independent score on this test
Gemini 4 ArgonOct 2026no independent score on this test
DeepSeek V4.1 Flash (Oct 2026)Oct 2026no independent score on this test
GPT-6.1 TerraOct 2026no independent score on this test
Gemini 4 CarbonOct 2026no independent score on this test
Kimi K3.1 ProNov 2026no independent score on this test
Claude Fable 5.2Nov 2026Latest releaseno independent score on this test
No published human baseline for this test. Bars use a log scale (each step is 10×), so small values stay visible. Click a score to open its source. * = reported by the lab, not independently measured. Source: METR.
Choose models
Models newest first. Faded models have no score on the selected test.
OpenAI0 of 15latest Oct 2026
Anthropic0 of 17latest Nov 2026
Google0 of 8latest Oct 2026
xAI0 of 2latest Sept 2026
DeepSeek0 of 2latest Oct 2026
Meta0 of 2latest Oct 2026
Moonshot0 of 2latest Nov 2026
Alibaba0 of 2latest Aug 2026
Who leads right now
The top model on each independent leaderboard. Leaderboards lag new releases by days to weeks.
| Benchmark | Leader | Score | As of |
|---|---|---|---|
| LMArena TextOverall, by human preference | Gemini 4 Carbon (High) | 1526.6 Arena scoreLMArena's text leaderboard, where people vote blind between two answers. Gemini 4 Argon is second at 1522.0, inside the margin of error (1517–1536). Claude Fable 5.2 isn't listed yet. | 9 Nov 2026 |
| Epoch Capabilities IndexOverall, many tests combined | Claude Fable 5.2 | 174.2 pointsEpoch Capabilities Index, one score built from more than 50 tests (range 170.1–178.8, from 10 tests so far). GPT-6 Astra is second at 169.2. Gemini 4 Carbon isn't rated because Epoch has no access to it. | 9 Nov 2026 |
| Artificial Analysis Intelligence Index v5*Overall, about ten tests combined | Claude Fable 5.2 (max) | 64.6 pointsVersion 5.0.1 of the index. Gemini 4 Carbon (high) is second at 60.0, Claude Opus 5.5 (max) third at 59.1, GPT-6 Astra (max) fourth at 56.9. * The index was rebuilt on 27 October 2026, so these scores can't be compared with earlier ones. | 10 Nov 2026 |
| LMArena Code ArenaCoding, web apps by human preference | Claude Opus 5.5 (Max) | 1818.2 Arena scoreLMArena's Code Arena, where people vote between web apps built by two models. Claude Sonnet 5.5 (xHigh) is second at 1773.5, Gemini 4 Carbon third at 1768.1. | 8 Nov 2026 |
| MirrorCodeCoding, weeks-long tasks | Claude Opus 5.5 (high) | 78.9%Epoch AI's MirrorCode. Claude Fable 5.1 scores 73.3%, Gemini 4 Argon 58.9%, GPT-6 Astra 46.7%. Claude Fable 5.2 hasn't been run yet. | 22 Sept 2026 |
| SWE-Bench ProCoding, fixing real bugs | Claude Opus 5 (Claude Code harness, xhigh) | 99.4%Scale AI's SWE-Bench Pro, full public set (±0.4); 98.0% on the hard subset. Near the ceiling, and each lab runs its own harness. Opus 5.5 and Fable 5.2 aren't listed yet. | 6 Nov 2026 |
| Remote Labor IndexRemote work, paid freelance projects | Claude Fable 5.2 | 41.3%The share of real freelance projects finished to a standard a client would accept, from the Center for AI Safety and Scale AI. Claude Opus 5.5 is second at 21.3%, GPT-6 Astra third at 20.8%. GPT-5.2 scored 2.5%. | 10 Nov 2026 |
| FrontierMath Tiers 1–3Math, hard problems | Claude Fable 5.2 (max) | 95.1%Epoch AI's FrontierMath Tiers 1–3 (±1.3). GPT-6.1 Sol and GPT-6 Astra score 93.7% (±1.4), so the lead is inside the margin of error. On the research-level Tier 4, Fable 5.2 and GPT-6.1 Sol both score 100%. | 9 Nov 2026 |
| MathArena ArXivMathMath, problems from new papers | GPT-6.1 Sol (max) | 94.2%MathArena's ArXivMath: problems from recent papers (June, August and September 2026 sets), ±2.69. Claude Opus 5.5 (high) is second at 83.8%. | 6 Nov 2026 |
| OSWorld-VerifiedUsing a computer | Claude Fable 5 (1M) | 86.0%OSWorld-Verified: desktop tasks finished within 100 steps (85.96%, checked by the OSWorld team on 1 August 2026). One agent framework, “Intelligence-Indeed Agent”, scores 90.19%. Opus 5.5, GPT-6 and Fable 5.2 aren't listed. | 1 Aug 2026 |
| OSWorld 2.0Using a computer, long tasks | Claude Opus 5 (max) | 44.3%OSWorld 2.0: 108 longer desktop tasks, up to 500 steps each, scored pass or fail (44.33%). GPT-6, Gemini 4, Opus 5.5 and Fable 5.2 haven't been tested. | 15 Oct 2026 |
| METR time horizonAgents, task length | Claude Mythos Preview (early) | 17.4 hoursMETR's 50% time horizon: the task length a model finishes half the time (1,044.8 minutes; range about 8.5–55 hours). METR says results above 16 hours are unreliable and stopped updating on 8 September 2026, so no GPT-6, Claude 5.x or Gemini 4 model has a score. | 8 May 2026 |
| LMArena Agent ArenaAgents, human-rated tasks | Claude Opus 5.5 (Max) | 0.1996LMArena's Agent Arena score (range 0.179–0.220). Gemini 4 Carbon (High) is second at 0.166. | 8 Nov 2026 |
| GPQA DiamondScience, PhD-level questions | Gemini 4 Argon (high) | 96.4%Epoch AI's GPQA Diamond, PhD-level science questions. GPT-6 Astra (max) scores 95.8%, Claude Fable 5.2 93.9%. All are near the test's ceiling. | 9 Nov 2026 |
| ARC-AGI-2Visual puzzles | GPT-6 Astra (Max) | 95.0%ARC Prize verified, semi-private set, $1.12 per task. GPT-6.1 Sol (Max) scores 94.2% at $0.25 per task, Claude Fable 5.2 (XHigh) 93.8%. A human panel scores 100%. | 9 Nov 2026 |
| ARC-AGI-3Puzzle games | Claude Fable 5.2 (XHigh) | 81.2%ARC Prize verified, semi-private set, standard harness. GPT-6 Astra (Max) scores 62.7%. With each lab's own “provider adapter” harness, GPT-6 Astra leads at 99.9% and Fable 5.2 scores 97.7%. | 9 Nov 2026 |
Frontier model log
Notable models, newest first. Each name links to its announcement.
| Model | Lab | Released | Access | Notable scores |
|---|---|---|---|---|
| Claude Fable 5.2Latest releaseFrontierSame model as Mythos 5.2, with cyber and biology safeguards. $8 in / $40 out per million tokens, less than Fable 5.1. Leads Epoch's capability index (174.2) and ARC-AGI-3 (81.2%). | Anthropic | 9 Nov 2026 | Public | GPQA Diamond: 94%OTIS Mock AIME: 100%FrontierMath T1–3: 95% |
| Claude Mythos 5.2FrontierSame model as Fable 5.2 with fewer cyber and biology safeguards. Only for vetted cyber-defence and life-science organisations. | Anthropic | 9 Nov 2026 | Limited preview | none |
| GPT-6 CyberCybersecurity model for vulnerability reproduction and fixes; requires Daybreak Red approval. | OpenAI | 5 Nov 2026 | Limited preview | none |
| Kimi K3.1 ProPost-trained K3; 94.2% on Epoch GPQA Diamond; weights promised by 2026-11-04 (HF repo live). | Moonshot | 4 Nov 2026 | Public | GPQA Diamond: 94%OTIS Mock AIME: 100%FrontierMath T1–3: 78% |
| Mistral Large 41T-total / 49B-active multimodal MoE ('Le Chonk'); public preview 2026-10-06, open weights 2026-10-29. | Mistral | 29 Oct 2026 | Public | none |
| Gemini 4 CarbonFrontierGoogle's strongest model; access limited to cyber defenders/Fairwind partners (no public API); #1 on LMArena text (1527, style control) as of 2026-11-08. | 28 Oct 2026 | Limited preview | FrontierSWE V2: 61%APEX-Agents: 84%AA Intelligence Index*: 60 points | |
| GPT-6.1 TerraFrontierFast, efficient model for visual reasoning and computer use; ECI 167.8 (#4). | OpenAI | 26 Oct 2026 | Public | GPQA Diamond: 94%OTIS Mock AIME: 100%FrontierMath T1–3: 91% |
| DeepSeek V4.1 Flash (Oct 2026)Highest open-weights model on Epoch's ECI (159.3) as of 2026-11-09. | DeepSeek | 17 Oct 2026 | Public | GPQA Diamond: 93%OTIS Mock AIME: 99%FrontierMath T1–3: 71% |
| Muse Spark OpenMeta's return to open weights after closed Muse Spark: 816B-total / 42B-active MoE, custom license. | Meta | 15 Oct 2026 | Public | none |
| Gemini 4 ArgonFrontierFirst Gemini 4 model; trusted cyber defenders (Fairwind) first, GA 2026-10-13; #1 GPQA Diamond on Epoch (96.4%); ECI 166.5. | 13 Oct 2026 | Public | GPQA Diamond: 96%OTIS Mock AIME: 100%FrontierMath T1–3: 86% | |
| Muse WonderMeta's most capable models; Wonder via Meta Model API, Wonder Full for Muse Code subscribers; ECI 162.0, #3 LMArena text (style control). | Meta | 12 Oct 2026 | Public | OTIS Mock AIME: 99%FrontierMath T1–3: 82%FrontierMath T4: 71% |
| Claude Haiku 5.5Cheapest/fastest Claude 5.5 model; ECI 160.2. | Anthropic | 7 Oct 2026 | Public | GPQA Diamond: 90%Humanity's Last Exam: 30%OTIS Mock AIME: 99% |
| GPT-6.1 SolFrontierNear-Astra capability at one-fifth of Astra's API price; ECI 168.9; 100% on Epoch FrontierMath Tier 4 v2. | OpenAI | 29 Sept 2026 | Public | GPQA Diamond: 95%OTIS Mock AIME: 100%FrontierMath T1–3: 94% |
| Claude Sonnet 5.5FrontierFaster, cheaper complement to Opus 5.5; ECI 166.2. | Anthropic | 28 Sept 2026 | Public | GPQA Diamond: 96%Humanity's Last Exam: 46%OTIS Mock AIME: 100% |
| Claude Opus 5.5FrontierFirst Claude 5.5 model; ~Fable 5.1-level at 40% lower cost than Opus 5; #1 LMArena Code Arena and Agent Arena; ECI 168.4. | Anthropic | 22 Sept 2026 | Public | GPQA Diamond: 91%OTIS Mock AIME: 100%FrontierMath T1–3: 91% |
| GPT-6 Sol and GPT-6 LunaLower-cost GPT-6 tiers (Sol $2/$10, Luna $0.10/$0.50 per 1M tokens). | OpenAI | 22 Sept 2026 | Public | none |
| Grok 4.7Latest xAI model (500K context); Grok 5 has not shipped (x.ai/news/grok-5 returns 404; release notes end at 4.7). | xAI | 21 Sept 2026 | Public | GPQA Diamond: 93%OTIS Mock AIME: 98%FrontierMath T1–3: 53% |
| GPT-6 AstraFrontierOpenAI's strongest public model. Second on the Epoch Capabilities Index (169.2) behind Claude Fable 5.2; first OpenAI model rated Critical for cybersecurity under its Preparedness Framework; ARC-AGI-2 95%, ARC-AGI-3 62.7% (standard harness). | OpenAI | 3 Sept 2026 | Public | GPQA Diamond: 96%Humanity's Last Exam: 55%OTIS Mock AIME: 100% |
| Gemini 3.8 FlashLatest Flash (with 3.8 Flash Cyber); ECI 158.6; Gemini 2.5 Pro requests reroute to it from 2026-12-01. | 2 Sept 2026 | Public | GPQA Diamond: 95%Humanity's Last Exam: 45%OTIS Mock AIME: 99% | |
| Claude Fable 5.1Fable upgrade (~25% cheaper for typical workloads); ECI 166.9. Anthropic's page shows only 'September 2026'; day from OpenRouter/Epoch. | Anthropic | 1 Sept 2026 | Public | Humanity's Last Exam: 47%OTIS Mock AIME: 100%FrontierMath T1–3: 90% |
| Claude Mythos 5.1Same model as Fable 5.1 with different safeguards; trusted-access programs (cyber, life sciences) only. | Anthropic | 1 Sept 2026 | Limited preview | none |
| DeepSeek-V4 (Pro / Flash)Open weights with 1M context: V4-Pro 1.6T total / 49B active, V4-Flash 284B / 13B; preview 2026-04-24, V4-Pro GA 2026-08-13. | DeepSeek | 13 Aug 2026 | Public | none |
| Grok 4.6Focus on long-running agents; ECI 157.9. | xAI | 12 Aug 2026 | Public | none |
| Qwen3.8 Max (0902)First open-weights Max-tier Qwen (2.4T total / 95B active as Qwen3.8-2.4T-A95B); hosted first, weights ~a week later. Epoch lists release 2026-08-02. | Alibaba | 12 Aug 2026 | Public | GPQA Diamond: 92%OTIS Mock AIME: 100%FrontierMath T1–3: 66% |
| Kimi K32.8T-parameter open-weights model ('first open 3T-class model'); API/apps 2026-07-16, weights by 2026-07-27. | Moonshot | 27 Jul 2026 | Public | none |
| Claude Opus 5Close to Fable 5 at half the price; tops Scale's SWE-Bench Pro V2 (99.4%) and OSWorld 2.0 (44.3% binary). | Anthropic | 24 Jul 2026 | Public | GPQA Diamond: 94%OTIS Mock AIME: 99%FrontierMath T1–3: 86% |
| Gemini 3.6 FlashMore token-efficient Flash; launched with 3.5 Flash-Lite and 3.5 Flash Cyber. | 21 Jul 2026 | Public | none | |
| Grok 4.5Coding/agentic model at $2/$6 per 1M tokens; xAI now publishes as 'SpaceXSI'. | xAI | 16 Jul 2026 | Public | none |
| GPT-5.6 SolIntroduced Sol (frontier) / Terra (balanced) / Luna (efficient) tiers; limited preview to trusted partners at US government request before GA on 2026-07-09. | OpenAI | 9 Jul 2026 | Public | GPQA Diamond: 94%OTIS Mock AIME: 100%FrontierMath T1–3: 89% |
| Claude Sonnet 5Agentic Sonnet (plans, uses browsers/terminals, runs autonomously). | Anthropic | 30 Jun 2026 | Public | none |
| Claude Fable 5First generally available 'Mythos-class' model; access suspended 2026-06-12 under a US export-control directive and restored 2026-07-01. Top general model on OSWorld-Verified (86.0%). | Anthropic | 9 Jun 2026 | Public | GPQA Diamond: 86%OTIS Mock AIME: 100%FrontierMath T1–3: 87% |
| Claude Mythos 5Less-restricted Mythos-class sibling of Fable 5; trusted-access programs only. | Anthropic | 9 Jun 2026 | Limited preview | none |
| Claude Opus 4.8Opus upgrade at the same price; launched with user-selectable effort in claude.ai. | Anthropic | 28 May 2026 | Public | GPQA Diamond: 91%OTIS Mock AIME: 98%FrontierMath T1–3: 80% |
| Gemini 3.5 FlashGemini 3.5 family debut; Google promised 3.5 Pro 'next month' but no 3.5 Pro appears in the Gemini API release notes. | 19 May 2026 | Public | none | |
| GPT-5.5Frontier model for complex professional work; GPT-5.5 Pro alongside; API release 2026-04-24. | OpenAI | 24 Apr 2026 | Public | GPQA Diamond: 94%OTIS Mock AIME: 100%FrontierMath T1–3: 85% |
| Claude Opus 4.7Opus upgrade; top score in Epoch's SWE-bench Verified runs (83.5%). | Anthropic | 16 Apr 2026 | Public | GPQA Diamond: 90%Humanity's Last Exam: 36%OTIS Mock AIME: 98% |
| Muse SparkFirst model from Meta Superintelligence Labs; closed weights; powers Meta AI apps, API only as private preview for select partners. | Meta | 8 Apr 2026 | App only | none |
| Claude Mythos PreviewWithheld from general release; gated to Project Glasswing partners for defensive cyber work. Highest METR time horizon ever published (early version, p50 ~17.4 h; METR flags >16 h as unreliable). | Anthropic | 7 Apr 2026 | Limited preview | METR time horizon: 17 h |
| GPT-5.4Frontier model for professional work with 1M-token context and native computer use; GPT-5.4 Pro alongside. | OpenAI | 5 Mar 2026 | Public | GPQA Diamond: 93%Humanity's Last Exam: 36%OTIS Mock AIME: 98% |
| GPT-5.3-CodexAgentic coding model; launched in Codex 2026-02-05, Responses API 2026-02-24. | OpenAI | 24 Feb 2026 | Public | none |
| Gemini 3.1 ProPreview Pro model (gemini-3.1-pro-preview); METR p50 ~6.4 h; last Pro-branded Gemini before Gemini 4. | 19 Feb 2026 | Public | GPQA Diamond: 94%Humanity's Last Exam: 46%OTIS Mock AIME: 96% | |
| Claude Sonnet 4.6Sonnet with 1M-token context; 72.1% on OSWorld (OSWorld site via Epoch). | Anthropic | 17 Feb 2026 | Public | none |
| Qwen3.5 (397B-A17B)Open-weights flagship of the Qwen3.5 series. | Alibaba | 16 Feb 2026 | Public | none |
| Claude Opus 4.6First Opus with a 1M-token context window (beta); METR p50 time horizon ~12 h (95% CI ~5-60 h). | Anthropic | 5 Feb 2026 | Public | GPQA Diamond: 91%Humanity's Last Exam: 34%OTIS Mock AIME: 94% |
| GPT-5.2Flagship for professional work; METR p50 time horizon ~5.9 h; GPT-5.2 Pro released alongside. | OpenAI | 11 Dec 2025 | Public | GPQA Diamond: 91%Humanity's Last Exam: 28%OTIS Mock AIME: 96% |
| Mistral Large 3 (Mistral 3)Open-weights Mistral 3 family (Large 3 + Ministral 3). | Mistral | 2 Dec 2025 | Public | none |
| DeepSeek-V3.2Last major release before V4. | DeepSeek | 1 Dec 2025 | Public | none |
| Claude Opus 4.5Adds selectable effort levels in the API; METR p50 time horizon ~4.9 h. | Anthropic | 24 Nov 2025 | Public | GPQA Diamond: 86%Humanity's Last Exam: 25%OTIS Mock AIME: 86% |
| Gemini 3 ProFirst Gemini 3 model (gemini-3-pro-preview); METR p50 time horizon ~3.7 h. | 18 Nov 2025 | Public | GPQA Diamond: 93%Humanity's Last Exam: 38%OTIS Mock AIME: 91% | |
| Grok 4.1Grok 4 refresh; Grok 4.1 Fast later added to the API. | xAI | 17 Nov 2025 | Public | none |
| GPT-5.1ChatGPT upgrade (Instant/Thinking) on 2025-11-12; API release 2025-11-13. | OpenAI | 13 Nov 2025 | Public | none |
| Kimi K2 ThinkingOpen-weights reasoning version of K2 (date = OpenRouter listing; HF repo created 2025-11-04). | Moonshot | 6 Nov 2025 | Public | GPQA Diamond: 84%OTIS Mock AIME: 83% |
| Claude Sonnet 4.5Coding/computer-use Sonnet; 62.9% OSWorld (per OSWorld site via Epoch). | Anthropic | 29 Sept 2025 | Public | none |
| GPT-5OpenAI's 'best AI system yet' at launch; API family gpt-5 / mini / nano released same day. METR p50 time horizon ~3.4 h. | OpenAI | 7 Aug 2025 | Public | GPQA Diamond: 86%Humanity's Last Exam: 25%OTIS Mock AIME: 91% |
| Claude Opus 4.1Incremental Opus upgrade. | Anthropic | 5 Aug 2025 | Public | GPQA Diamond: 77%Humanity's Last Exam: 12%OTIS Mock AIME: 69% |
| gpt-oss-120bOpenAI open-weights reasoning models (gpt-oss-120b and gpt-oss-20b). | OpenAI | 5 Aug 2025 | Public | none |
| Qwen3-235B-A22B-Thinking-2507 | Alibaba | 25 Jul 2025 | Public | GPQA Diamond: 80%OTIS Mock AIME: 87% |
| Kimi K2Open-weights agentic MoE model ('Open Agentic Intelligence'). | Moonshot | 11 Jul 2025 | Public | none |
| Grok 4Billed by xAI as 'the most intelligent model in the world'; SuperGrok/Premium+ and API at launch. | xAI | 9 Jul 2025 | Public | GPQA Diamond: 87%OTIS Mock AIME: 84%ARC-AGI-2: 16% |
| Gemini 2.5 ProGemini 2.5 with thinking built in; experimental 2025-03-25, stable GA 2025-06-17. | 17 Jun 2025 | Public | GPQA Diamond: 85%Humanity's Last Exam: 22%OTIS Mock AIME: 85% | |
| Claude Opus 4Claude 4 generation (Opus 4 + Sonnet 4) aimed at coding, advanced reasoning and agents. | Anthropic | 22 May 2025 | Public | GPQA Diamond: 76%Humanity's Last Exam: 11%OTIS Mock AIME: 64% |
| Qwen3Open-weights Qwen3 family with thinking-mode fusion. | Alibaba | 29 Apr 2025 | Public | none |
| o3Preview set a breakthrough ARC-AGI-1 high score (Dec 2024); ARC Prize says the o3 released in April 2025 is not the same version it tested. | OpenAI | 16 Apr 2025 | Public | GPQA Diamond: 82%Humanity's Last Exam: 20%OTIS Mock AIME: 84% |
| o4-mini | OpenAI | 16 Apr 2025 | Public | GPQA Diamond: 80%Humanity's Last Exam: 18%OTIS Mock AIME: 82% |
| Llama 4 MaverickOpen-weights MoE models (Scout 109B total, Maverick 400B total, 17B active); Llama 4 Behemoth previewed but never released. | Meta | 5 Apr 2025 | Public | GPQA Diamond: 67%Humanity's Last Exam: 5.7%OTIS Mock AIME: 21% |
| GPT-4.5Research preview billed as OpenAI's 'largest and most knowledgeable model yet' (non-reasoning). | OpenAI | 27 Feb 2025 | Public | none |
| Claude 3.7 SonnetBilled as the first hybrid reasoning model (instant or extended thinking); launched with Claude Code. | Anthropic | 24 Feb 2025 | Public | GPQA Diamond: 80%Humanity's Last Exam: 8%OTIS Mock AIME: 58% |
| Grok 3'Grok 3 Beta' reasoning model (date per xAI post); API launched April 2025. | xAI | 19 Feb 2025 | Public | none |
| Gemini 2.0 FlashStart of the Gemini 2.0 'agentic era'; experimental release Dec 2024, GA 2025-02-05. | 5 Feb 2025 | Public | GPQA Diamond: 64%OTIS Mock AIME: 31% | |
| DeepSeek-R1Open-weights (MIT) reasoning model; DeepSeek claimed performance on par with OpenAI o1 (self-reported). | DeepSeek | 20 Jan 2025 | Public | GPQA Diamond: 72%OTIS Mock AIME: 53%ARC-AGI-2: 1.3% |
| DeepSeek-V3Open-weights MoE chat model. | DeepSeek | 26 Dec 2024 | Public | none |
| o1First o-series reasoning model (RL-trained to reason before answering); o1-preview/o1-mini 2024-09-12, full o1 system card 2024-12-05, o1 in API 2024-12-17. | OpenAI | 17 Dec 2024 | Public | GPQA Diamond: 77%Humanity's Last Exam: 8%OTIS Mock AIME: 73% |
| GPT-4o (Nov 2024) | OpenAI | 20 Nov 2024 | Public | GPQA Diamond: 48%Humanity's Last Exam: 2.7%OTIS Mock AIME: 6.3% |
| Claude 3.5 Sonnet (Oct 2024) | Anthropic | 22 Oct 2024 | Public | GPQA Diamond: 55%Humanity's Last Exam: 4.1%OTIS Mock AIME: 8.5% |
| Claude 3.5 SonnetMid-tier model that outperformed Claude 3 Opus; model ID claude-3-5-sonnet-20240620 (Anthropic's page is dated Jun 21, 2024). | Anthropic | 20 Jun 2024 | Public | none |
| Gemini 1.5 Pro (002)Long-context breakthrough: up to 1M tokens (private preview at launch); GA in the Gemini API 2024-05-23. | 23 May 2024 | Public | GPQA Diamond: 57%Humanity's Last Exam: 4.6%OTIS Mock AIME: 23% | |
| GPT-4o'Omni' model reasoning across audio, vision and text in real time; released in the API the same day. | OpenAI | 13 May 2024 | Public | GPQA Diamond: 49%OTIS Mock AIME: 6.3%METR time horizon: 7 min |
| GPT-4 Turbo (Apr 2024) | OpenAI | 9 Apr 2024 | Public | GPQA Diamond: 47%OTIS Mock AIME: 6.7%FrontierMath T1–3: 0.7% |
| Claude 3 OpusFlagship of the Claude 3 family (Opus/Sonnet/Haiku); 200K context; $15/$75 per 1M tokens. | Anthropic | 4 Mar 2024 | Public | GPQA Diamond: 47%OTIS Mock AIME: 4.7%METR time horizon: 4 min |
| GPT-4Defined the 2023 frontier; large multimodal model accepting image and text inputs (per technical report). | OpenAI | 14 Mar 2023 | Public | GPQA Diamond: 36%OTIS Mock AIME: 0.6%METR time horizon: 4 min |