Same AI, same puzzle games: 62.7% with the standard test setup, 99.9% with its maker's own
How a model is run in a test can matter as much as the model. GPT-6 Astra scored 62.7% or 99.9% depending on the software wrapped around it.
Updated 10 Nov 2026: Added Claude Fable 5.2, the new best score with the standard harness (81.2%).
ARC-AGI-3: interactive puzzle games
Best ARC Prize score on the semi-private set. Standard harness vs a lab's own harness.
What this measures: Hundreds of small turn-based games with no instructions: the AI has to explore, work out the rules and win in as few moves as possible. Every game was beaten by human testers; 100% means matching a typical first-time human player's efficiency on every game.
- ARC's standard harness
- Lab's own harness (provider adapter)
- Reference level
The harness matters on interactive tasks: every model tested both ways scored far higher when it could keep its own hidden reasoning between moves (e.g. GPT-6 Astra 62.7% → 99.9%, Claude Fable 5.2 81.2% → 97.7%). Dates are model release dates. ARC-AGI-3 costs are per run, not per task. ARC Prize 2026 competition results are due on 4 December 2026.
Show data table and sources
| Series | Date | Model | Value |
|---|---|---|---|
| ARC's standard harness | 25 Mar 2026 | Best frontier AI at launch | 0.5% |
| ARC's standard harness | 1 Jun 2026 | Claude Opus 4.8 (high) | 1.5% |
| ARC's standard harness | 9 Jul 2026 | GPT-5.6 Sol (max) | 7.8% |
| ARC's standard harness | 24 Jul 2026 | Claude Opus 5 (high) | 30% |
| ARC's standard harness | 2 Sept 2026 | GPT-6 Astra (max) | 63% |
| ARC's standard harness | 2 Sept 2026 | Gemini 3.8 Flash | 10% |
| ARC's standard harness | 28 Sept 2026 | Claude Sonnet 5.5 | 24% |
| ARC's standard harness | 29 Sept 2026 | GPT-6.1 Sol | 53% |
| ARC's standard harness | 30 Sept 2026 | Gemini 4 Argon | 50% |
| ARC's standard harness | 9 Nov 2026 | Claude Fable 5.2 | 81% |
| Lab's own harness (provider adapter) | 2 Sept 2026 | Gemini 3.8 Flash + adapter | 35% |
| Lab's own harness (provider adapter) | 3 Sept 2026 | GPT-6 Astra + OpenAI adapter | 100% |
| Lab's own harness (provider adapter) | 28 Sept 2026 | Claude Sonnet 5.5 + adapter | 72% |
| Lab's own harness (provider adapter) | 29 Sept 2026 | GPT-6.1 Sol + adapter | 96% |
| Lab's own harness (provider adapter) | 30 Sept 2026 | Gemini 4 Argon + adapter | 98% |
| Lab's own harness (provider adapter) | 9 Nov 2026 | Claude Fable 5.2 + adapter | 98% |
| Reference | Typical first-time human player (100%) | 100% |
- ARC-AGI Leaderboard (ARC Prize)
- ARC Prize leaderboard data (evaluations.json) (ARC Prize)
- ARC-AGI-3 launch (ARC Prize)
- GPT-6 Astra results (ARC Prize)
- OpenAI's GPT-6 Astra on ARC-AGI-3 (ARC Prize Foundation)
ARC-AGI-2 puzzles: with and without a harness
Best ARC Prize-verified score on the semi-private set, by model release date.
What this measures: Visual logic puzzles that every task was solved by at least two people, but that AI found almost impossible in early 2025. A "harness" is extra code around the model that lets it try, check and refine answers.
- Model alone (ARC's standard setup)
- With a third-party refinement harness
- Reference level
Model alone (ARC's standard setup), Saturated: The best model scores 95% (September 2026), and ARC Prize says ARC-AGI-1 and -2 are being overfit. Its focus has moved to the interactive ARC-AGI-3. Now watch: ARC-AGI-3.
With a third-party refinement harness, No new results · last data Feb 2026: No new verified harness result since February 2026: plain models overtook the harness systems (GPT-6 Astra scored 95% at $1.12 per task, versus 72.9% at $38.99 for the best harness), so there has been little reason to submit new ones.
Harness systems led in 2025, then plain models overtook them in 2026. The compute-capped ARC Prize 2025 Kaggle competition was won by NVARC with 24.0% at about $0.20 per task. ARC Prize has said ARC-AGI-1 and -2 are being overfit, intentionally or not.
Show data table and sources
| Series | Date | Label | Value |
|---|---|---|---|
| Model alone (ARC's standard setup) | 20 Dec 2024 | o3-preview (low) | 4% |
| Model alone (ARC's standard setup) | 16 Apr 2025 | o3 (high) | 6.5% |
| Model alone (ARC's standard setup) | 14 May 2025 | Claude Opus 4 | 8.6% |
| Model alone (ARC's standard setup) | 9 Jul 2025 | Grok 4 (thinking) | 16% |
| Model alone (ARC's standard setup) | 6 Oct 2025 | GPT-5 Pro | 18% |
| Model alone (ARC's standard setup) | 18 Nov 2025 | Gemini 3 Pro | 31% |
| Model alone (ARC's standard setup) | 24 Nov 2025 | Claude Opus 4.5 | 38% |
| Model alone (ARC's standard setup) | 11 Dec 2025 | GPT-5.2 Pro (high) | 54% |
| Model alone (ARC's standard setup) | 5 Feb 2026 | Claude Opus 4.6 (high) | 69% |
| Model alone (ARC's standard setup) | 12 Feb 2026 | Gemini 3 Deep Think | 85% |
| Model alone (ARC's standard setup) | 22 Apr 2026 | GPT-5.5 (xhigh) | 85% |
| Model alone (ARC's standard setup) | 9 Jun 2026 | Claude Fable 5 (max) | 89% |
| Model alone (ARC's standard setup) | 9 Jul 2026 | GPT-5.6 Sol (max) | 93% |
| Model alone (ARC's standard setup) | 2 Sept 2026 | GPT-6 Astra (max) | 95% |
| With a third-party refinement harness | 7 Aug 2025 | Grok 4 + refinement (j_berman) | 29% |
| With a third-party refinement harness | 4 Dec 2025 | Gemini 3 Pro + Poetiq | 54% |
| With a third-party refinement harness | 3 Feb 2026 | GPT-5.2 + refinement (johan_land) | 73% |
| Reference | Human panel (100%) | 100% | |
| Reference | Average test-taker (60%) | 60% |
- ARC-AGI Leaderboard (ARC Prize)
- ARC Prize leaderboard data (evaluations.json) (ARC Prize)
- Announcing ARC-AGI-2 and ARC Prize 2025 (ARC Prize)
- ARC Prize 2025 Results & Analysis (ARC Prize)
A harness is the software wrapped around a model during a test. It decides what the model sees, what it remembers between steps, whether it can run code and whether it gets several checked attempts. ARC Prize reports results under different harnesses separately.
ARC-AGI-3 is a set of puzzle games with no instructions. With ARC Prize’s minimal standard harness, the best score went from 0.51% at launch (March 2026) to 30.2% (Claude Opus 5, July), 62.7% (GPT-6 Astra, September) and 81.2% (Claude Fable 5.2, November). OpenAI’s provider adapter lets the model keep its hidden reasoning between moves and condenses long game histories. With it, GPT-6 Astra scored 99.9%, using fewer moves than the median human on 96% of levels. Every model tested both ways improved: Claude Fable 5.2 from 81.2% to 97.7%, Gemini 4 Argon from 49.8% to 97.5%, GPT-6.1 Sol from 52.7% to 96.4% and Claude Sonnet 5.5 from 24.3% to 72.4%.
ARC-AGI-2 (visual puzzles) shows the extra software mattering less over time. In 2025, third-party “refinement” harnesses, which generate and check many candidate answers, led the leaderboard. They peaked at 72.9% for $38.99 per task in February 2026. Then plain models overtook them: Gemini 3 Deep Think 84.6%, GPT-5.6 Sol 92.5% and GPT-6 Astra 95.0% at $1.12 per task.
What it means
A test score describes a model and its setup together, not the model alone. Before comparing two headline numbers, check that they used the same harness: on ARC-AGI-3 the setup moved one model from 62.7% to 99.9%. As models improve, clever add-ons can also stop paying off, as happened on ARC-AGI-2.
Caveats
- ARC Prize says that topping ARC-AGI-3 is not proof of AGI (AI that matches people across the board): its games are deterministic (the same move always has the same effect) and closed-ended. ARC Prize 2026 results are due 4 December 2026.
- ARC-AGI-3 costs are reported per run, not per task, and some runs go over ARC Prize’s stated $10,000 cap without explanation.
- ARC Prize suspects ARC-AGI-1 and -2 are being overfit, with models tuned to the puzzles through prior knowledge.