AI chips' capacity to run AI agents grew 4.6× in a year, against 2.6× for raw compute
Epoch AI says newer Nvidia Blackwell chips do much better on long, cached workloads than their raw compute suggests.
Epoch AI estimated how much AI work all the chips sold so far can do. Between Q2 2025 and Q2 2026, that capacity for agentic inference grew 4.6×. Inference means running a trained model to answer requests, and agentic workloads are the long ones with large amounts of cached context (text the model keeps in memory between steps), which is how AI agents run. Peak 8-bit computing power, the usual measure of a chip’s raw arithmetic speed, grew 2.6× over the same year. Epoch’s 90% uncertainty intervals are 4.2–5.1× and 2.5–2.7×.
Epoch measures capacity with its Inference Performance Index, which scores each chip on a fixed set of agentic workloads and accounts for memory, links between chips and lower-precision arithmetic. It multiplies each chip’s score by the number sold, using the estimates in Epoch’s AI Chip Sales dataset. Capacity per unit of peak compute rose about 77% over the year.
Nvidia’s Blackwell and Blackwell Ultra chips account for 74% of the capacity added. Epoch models a GB300 system as delivering about 8× the capacity of an H200 per GPU. By Q2 2026 those two families held about 73% of capacity but 50% of peak compute.
Epoch expects the pace to slow. Chips sold in Q2 2026 added 3.3× the capacity of those sold in Q2 2025, below the 4.6× growth in the total. It models the step from GB300 to Nvidia’s next chip, Vera Rubin VR200, at only 2.2× per GPU.
What it means
Raw computing power understates how much more AI agent work the world’s chips can serve than a year ago. Epoch counts hardware capacity here, not prices or usage, so this entry does not show that running an agent costs less per task.
Caveats
- The capacity figures combine estimates of chips sold with Epoch’s own modelled per-chip scores. The uncertainty intervals cover shipment and hardware-mix errors, not errors in the modelled performance.
- The index is Epoch’s own, version 1, and uses one fixed set of agentic workloads for every quarter. Real workloads differ, and software efficiency is not modelled.
- Epoch assumes 60% of paired Blackwell GPUs are rack systems and 40% are servers, and splits H100 and H200 sales evenly from Q4 2024. It calls the first a judgment-based assumption.
- Capacity installed is not capacity used, and the estimate counts chips sold to date, not chips in service.