A new test gives AI agents $200 to train models: the best scores 58.8, a human researcher 66.1
FrontierML from Induction Labs has agents buy computing and data for 21 machine-learning tasks. A bigger budget helped the top models most.
FrontierML, published by Induction Labs, asks how well an AI agent turns money into a better machine-learning model. It has 21 long tasks in six areas, from training language models to speech, vision, computer use and math. Each agent gets a 12-hour sandbox with no GPU, a few validation samples and a wallet of $10, $50 or $200. It pays for GPUs and AI tokens from the wallet. The result is an inference server that runs offline on an H100 and is scored on held-out test sets the agent never saw.
At $200, Claude Opus 5.5 scores 58.8 and the human researcher entry 66.1. At $10 the two score 45.4 and 56.0. Claude Sonnet 5.5 is next at 50.4, then GPT-6 Astra at 44.5. The best open-weight model, Kimi K3.1 Pro, scores 25.5 at $10 and 29.6 at $200. Opus 5.5 and Sonnet 5.5 gained the most from the extra money, while GPT-6 Luna and Gemini 3.8 Flash did not improve.
Agents added newly trained parts to a pretrained model, or trained a model from scratch, in 28% of runs. They spent about 45% of their wallet on average and none passed 80%. Of 735 training runs, 14% used only generated data and 35% only public data.
What it means
An agent that can rent computing and turn it into a better model is doing a small piece of an AI researcher’s job. On this test the two Claude agents improved when given more money, and the human entry stayed ahead of them at both budgets shown.
Caveats
- Induction Labs built the test and ran it. Claude Opus 5.5, the top-scoring model, also reviews every submission for cheating, and a failed review scores 0.
- The page gives no details on the human researcher entry: how many people, how long they worked or what they spent. It also does not say what the score is measured against.
- There were 1,197 runs in total, but the page does not break them down per model, task or budget.
- Each agent ran at one reasoning effort, and GPU prices are fixed at a 29 September 2026 snapshot. The authors say some tasks are too hard at these budgets and plan budgets up to $100,000.
- There is no paper. The results have not been repeated by anyone else.