Charts

What this measures: Unlike exam-style tests, nobody knew the answers to these problems when the test was made. A model only scores if the Lean proof checker accepts its proof or disproof.

Plotted at model release dates (the benchmark launched 1 September 2026, so earlier models were evaluated retroactively). Outside the official one-attempt rules, 17 of the 68 problems have now been solved by some means, including 12 credited to OpenAI's 6 October research release. Contamination risk grows as solutions are published.

What this measures: Paul Erdős left behind more than a thousand math problems. This counts how many have a solution that the Lean proof checker has verified step by step, whether the proof came from a human or an AI.

Counts solutions from any source, human or AI, so this is not a count of AI solutions. Early growth partly reflects the database being built. The sources don't explain the August 2026 jump. Since 6 October 2026 erdosproblems.com no longer shows solved counts, and the 12 FrontierMath Erdős problems Epoch AI credits to OpenAI's 6 October release are not yet included. Each point is the last row of its month in the database's statistics_history.csv.

What this measures: Problems from the level of the American Invitational Mathematics Examination, the qualifier for the US Math Olympiad team. In 2023 the best AI solved almost none.

Saturated · last data Apr 2026: Top models score 100% (GPT-5.5 Pro, April 2026), so this test can no longer show progress and labs stopped reporting it. Now watch: FrontierMath and FrontierMath Erdős (open problems).

Release dates for GPT-4 (0613), Claude 2.0, Claude 3 Opus and Llama 3.1 are the models' public release dates. Each problem is worth about 2.2 points. Labs stopped reporting AIME in 2026 because top models solve it all.

What this measures: The International Mathematical Olympiad is the world's top math contest for high-school students: six proof problems, 42 points in total. Gold usually needs about 29–35.

• Best AI score
• Reference level

Saturated: AI systems were reported to reach a perfect 42/42 in 2026, so the contest can't show further progress. The next frontier is open research problems. Now watch: FrontierMath Erdős and Erdős problems solved.

Gold-medal cutoffs: 35 (2025), 29 (2026). The 2026 results were reported by Chinese media and the companies; we found no statement published by the IMO itself.

What this measures: Hundreds of unpublished problems written by mathematicians. Tiers 1–3 are advanced; Tier 4 is research-level. Answers are checked automatically.

• Tiers 1–3 (advanced)
• Tier 4 (research-level)

Tier 4 (research-level), Saturated: Epoch AI considers Tier 4 saturated: GPT-6 Astra solved the last problem no model had solved (September 2026), and GPT-6.1 Sol scores 100%. Now watch: FrontierMath Erdős (68 genuinely open problems).

Epoch AI re-ran older models on the corrected v2 problem set (June 2026), so early points come from those re-runs. OpenAI funded FrontierMath and has access to many problems; Epoch keeps a held-out set. Answers are checked automatically. These are not proofs of open problems.

What this measures: The AI gets a real bug report from a popular open-source project and must change the code so the project's own tests pass.

• Independent (Epoch AI, fixed setup)
• Self-reported by labs

Independent (Epoch AI, fixed setup), Known flaws · last data Apr 2026: Epoch AI stopped adding runs after June 2026, and OpenAI stopped reporting this test in February 2026 after finding flawed tests and signs that models had memorised the fixes. Top scores were also near the ceiling. Now watch: SWE-bench Pro (Scale AI) and MirrorCode.

Self-reported by labs, Replaced by a newer test · last data Apr 2026: Labs largely stopped reporting SWE-bench Verified in 2026 after OpenAI's audit found flawed tests and memorised fixes; Anthropic's last reported score was 93.9% (April 2026). Now watch: SWE-bench Pro (Scale AI).

Epoch AI stopped adding SWE-bench runs after June 2026, and OpenAI stopped reporting the benchmark in February 2026 after finding flawed tests and signs that models had memorised fixes. Lab numbers use their own agent setups and run higher; the successor benchmark is SWE-bench Pro.

What this measures: A medical device can only be sold in the United States after the FDA clears or approves it. The agency publishes a list of the ones that use artificial intelligence. Most are software that reads medical scans, not chatbots.

The last point covers only the first half of 2026. The FDA's list ran to decisions made up to 29 June 2026 when it was last updated on 4 September 2026, so the 2026 total will rise.

Counted from the FDA's CSV of the list (file dated 4 September 2026): 1,614 authorisations. 1,553 used the 510(k) route, which asks whether a device matches one already on the market, and 1,230 are radiology devices. Some products appear more than once as later versions were authorised (1,317 distinct device names from 817 companies). The FDA says the list is not comprehensive.

What this measures: Each line is a test built to be hard for AI. Watch how quickly each one climbs from near zero toward the top once it exists.

• GPQA Diamond
• FrontierMath Tiers 1–3
• Humanity's Last Exam
• ARC-AGI-2
• ARC-AGI-3

GPQA and FrontierMath points are Epoch AI's independent runs (FrontierMath on the corrected 2026 set, re-run on older models); HLE is Scale AI's no-tools leaderboard; ARC points are ARC Prize-verified with the standard setup. Fast saturation partly reflects benchmark flaws and contamination, not only skill.

What this measures: Artificial Analysis combines about ten tests of reasoning, coding, knowledge work and agent tasks into one score out of 100. It is one of the most quoted single numbers for how capable a model is.

• Version 4.0 (January to June 2026)
• Versions 4.1 to 4.3 (June to October 2026)
• Version 5 (current)

* The index is rebuilt every few months with harder tests, so a score only means something next to other scores from the same version. Claude Fable 5.1 scored 66 on 1 September 2026 and 53 six days later, after two updates. The current version is 5.0.1 (3 November 2026).

Version 4.0 (January to June 2026), Replaced by a newer test · last data Jun 2026: Replaced by version 4.1 on 15 June 2026, which swapped several tests. Claude Fable 5 went from 64.9 to 60 overnight. Now watch: version 5.

Versions 4.1 to 4.3 (June to October 2026), Replaced by a newer test: Four updates in four months added harder, partly private tests. The top score fell from 66 to 53 between 1 and 7 September 2026. Now watch: version 5.

Version 4.0 scores are Artificial Analysis's re-scoring of older models, read from its January 2026 leaderboard and later articles. Version 5 scores were read on 10 November 2026; models released before about July 2026 have no version 5 score. For comparing across time, Artificial Analysis publishes a separate Progress Index, on which GPT-4 scores 140 and Claude Fable 5.2 scores 358. Versions 1 to 3 (2024–2025) are left out.

What this measures: 2,500 expert-written questions chosen because the best AI of early 2025 got them wrong. At launch, top models scored under 10%.

Labs report higher scores with tools (e.g. 64.7% for Claude Mythos Preview, self-reported). Those are not comparable with these no-tools runs. Independent reviews estimate 18–29% of the biology/chemistry answers conflict with the literature, so scores above ~70–80% may reflect label noise rather than skill.

What this measures: Hundreds of small turn-based games with no instructions: the AI has to explore, work out the rules and win in as few moves as possible. Every game was beaten by human testers; 100% means matching a typical first-time human player's efficiency on every game.

• ARC's standard harness
• Lab's own harness (provider adapter)
• Reference level

The harness matters on interactive tasks: every model tested both ways scored far higher when it could keep its own hidden reasoning between moves (e.g. GPT-6 Astra 62.7% → 99.9%, Claude Fable 5.2 81.2% → 97.7%). Dates are model release dates. ARC-AGI-3 costs are per run, not per task. ARC Prize 2026 competition results are due on 4 December 2026.

What this measures: 198 hard biology, chemistry and physics questions written by PhDs. Experts in the field get about 70% right; smart non-experts with Google get 34%; guessing gets 25%.

• Best AI score (Epoch AI runs)
• Reference level

Saturated: Top scores (about 96%) are near the test's ceiling, where errors in the answer key cap what's achievable, so differences between today's best models are mostly noise. Now watch: Humanity's Last Exam.

Independent runs by Epoch AI, plotted at each model's release date (many runs happened later). The expert baseline comes from a study OpenAI ran with recruited PhDs; the original paper reports 65% on the full question set. Questions have been public since 2023, so training-data contamination can't be ruled out, and scores above ~95% may be near the benchmark's error floor.

What this measures: Visual logic puzzles that every task was solved by at least two people, but that AI found almost impossible in early 2025. A "harness" is extra code around the model that lets it try, check and refine answers.

• Model alone (ARC's standard setup)
• With a third-party refinement harness
• Reference level

Model alone (ARC's standard setup), Saturated: The best model scores 95% (September 2026), and ARC Prize says ARC-AGI-1 and -2 are being overfit. Its focus has moved to the interactive ARC-AGI-3. Now watch: ARC-AGI-3.

With a third-party refinement harness, No new results · last data Feb 2026: No new verified harness result since February 2026: plain models overtook the harness systems (GPT-6 Astra scored 95% at $1.12 per task, versus 72.9% at $38.99 for the best harness), so there has been little reason to submit new ones.

Harness systems led in 2025, then plain models overtook them in 2026. The compute-capped ARC Prize 2025 Kaggle competition was won by NVARC with 24.0% at about $0.20 per task. ARC Prize has said ARC-AGI-1 and -2 are being overfit, intentionally or not.

What this measures: AI 2027 is a detailed 2025 forecast in which AI that out-codes the best humans arrives inside a lab by March 2027. A bar longer than the dashed line means reality is moving faster than the scenario on that measure.

These grades stop in August 2026. Since then, METR's audit (5 November) found AI roughly doubling Anthropic's research pace, a level the scenario reaches around March 2027, so the “AI speeding up AI research” bar is out of date. GPT-6 Astra and Claude Fable 5.2 are not graded yet.

Paces are the AI Futures Project's own grading (Feb and Aug 2026) and are sensitive to how each metric is measured. They are the scenario authors' estimates, not independent ones.

What this measures: How big a job an AI agent can finish on its own. A task's "length" is how long it takes a skilled human; the chart shows the longest tasks an AI completes half the time (blue) or reliably (orange). Tasks are software, machine-learning and cybersecurity work.

• 50% success
• 80% success
• Exponential trend

Dashed line: exponential fit since 1 Jan 2023, doubling roughly every 4.0 months.

50% success, No longer updated · last data Apr 2026: METR stopped updating this measure on 8 September 2026. The newest models finish almost every task in its test suite, and METR says results above about 16 hours are unreliable, so its last data point is Claude Mythos Preview (April 2026), and no GPT-6, Claude 5.x or Gemini 4 model has a score. METR has moved to measuring how much AI speeds up AI research inside labs. Now watch: METR's AI R&D acceleration audits (e.g. Anthropic, Nov 2026) and Epoch AI/METR's MirrorCode weeks-long coding tasks.

80% success, No longer updated · last data Apr 2026: METR stopped updating this measure on 8 September 2026. The newest models finish almost every task in its test suite, and METR says results above about 16 hours are unreliable, so its last data point is Claude Mythos Preview (April 2026), and no GPT-6, Claude 5.x or Gemini 4 model has a score. METR has moved to measuring how much AI speeds up AI research inside labs. Now watch: METR's AI R&D acceleration audits (e.g. Anthropic, Nov 2026) and Epoch AI/METR's MirrorCode weeks-long coding tasks.

METR Time Horizon 1.1 values (after METR's March 2026 correction), plotted at model release dates. Tasks are well-specified software, ML and cybersecurity tasks. Confidence intervals are wide (e.g. Claude Opus 4.6: 317–3,634 min). METR stopped updating this dataset in September 2026 and says measurements above ~16 hours are unreliable with its current tasks, so models released after spring 2026 are not shown.

What this measures: What it costs to use an AI at least as capable as the original GPT-4 (2023). A million tokens is roughly 750,000 words of input and output.

Compare with other reference models →

2023–2024 points are Epoch AI's. Points from late 2024 on are AI Tracker's extension using archived OpenRouter price lists and published GPQA Diamond scores (one provider route, not always the cheapest). The price of this capability level has flattened at roughly $0.05–0.13 since late 2024; the steep declines are now at higher capability levels (see the ARC-AGI cost chart). Per-token price is not cost per task.

What this measures: How much it costs an AI to solve one ARC-AGI-1 visual logic puzzle while scoring 75% or more.

Dates are ARC Prize's model release dates. Costs depend on ARC Prize's pricing assumptions, which it has revised (o3-preview went from $26 to $200 per task). The tested o3-preview was trained on part of the public training set and differs from the released o3.

What this measures: A short IQ-style pattern quiz that has never been published online, so models can't have memorised it. 100 is the human average.

• Top model (offline test)
• Reference level

Monthly maxima, dated to the end of each month. IQ norms are designed for humans and aren't validated for AI; this is a small matrix-style quiz, not a measure of general intelligence. On the public Mensa Norway test (which may be in training data) scores are higher, about 150.

What this measures: How much water one AI text prompt uses, next to everyday activities and products.

Scopes differ, so compare like with like. The AI prompt, toilet and shower rows are direct use. Coffee, beef and jeans are supply-chain or life-cycle footprints (coffee and beef include rainwater taken up by crops and pasture); the fair AI comparison for those is the life-cycle row (Mistral, 45 mL). Long-document, reasoning and agentic requests use far more than a median text prompt.

What this measures: How much water US data centres use in a year, next to other big water users.

Annual values derived from published daily or litre figures. Years and metrics differ (consumption vs water applied vs withdrawal; USGS 2015 is the latest national compilation in this form). National totals hide local water stress, which is where data-centre water use matters most.

What this measures: How much electricity different AI requests use, next to everyday activities.

AI figures are company disclosures (Google, OpenAI) or published models (Microsoft-led Joule study, Epoch AI, IEA); none is independently audited. Household figures assume typical UK appliances (Hannah Ritchie). The web-search figure is Google's from 2009.

What this measures: How the world splits by how much AI people use, from those who have barely tried a chatbot to the few thousand people working inside AI labs.

The three grey bars are one researcher's rough estimate, which he calls “napkin math”. Nobody has measured these groups. The two blue bars are company figures from different months.

Karpathy's tiers describe the whole world (about 8 billion people) and are not a breakdown of ChatGPT's users; his shares add up to about 95%. OpenAI does not say how it counts or deduplicates weekly users, and its latest subscriber count (February 2026) is seven months older than its 1.2 billion figure, so the two should not be divided. Groups overlap: a paying subscriber is also a weekly user.

What this measures: How much it costs to buy the cheapest humanoid robot from Unitree, from its first humanoid in 2023 to today. These are prices for basic versions of small research and hobby robots, not for a robot that can do housework.

List prices for base models, converted to dollars at each source's rate; prices outside China are higher. The G1 and R1 are child-size (121–127 cm) and sold mainly to universities, labs and developers. What buyers paid fell too: Unitree's average selling price per humanoid dropped from about $85,000 in 2023 to about $25,000 in the first nine months of 2025, according to its IPO filing.

What this measures: How many humanoid robots makers sent to customers, according to four market trackers, plus two forecasts for 2026. For comparison, factories installed just over 600,000 conventional industrial robots in 2025.

A shipment is not a deployment. Most humanoids shipped so far went to research labs, data collection, and shows and exhibitions, not to paid work in factories or warehouses.

Definitions differ. The International Federation of Robotics counts only full-size humanoids (over 140 cm) sold for commercial and professional use. IDC, Omdia and Counterpoint also count smaller robots and those sold for research, education and entertainment. Some vendor figures are disputed: Unitree says it shipped more than 5,500 in 2025, Omdia estimated about 4,200. Chinese makers built more than 95% of the robots IDC counted in the first half of 2026.

What this measures: How long the fastest official robot took to run a half-marathon in Beijing's annual race for humanoid robots. Lower is faster.

• Winning robot
• Reference level

No new results · last data Apr 2026: The race is held once a year (both editions so far were in April), so the next result will come from the 2027 race.

In 2026 about 40% of the more than 100 robot entrants ran autonomously and about 60% were remote-controlled (Beijing E-Town, via TechCrunch). Robots ran in lanes separated from human runners. The fastest human in the 2025 race finished in 1:02:36. Running speed on a closed course says little about whether a robot can do useful work.