Mathematicians gave AI 10 unpublished research problems; the best system solved 6
Four AI systems tried 10 research problems that had never been published. Referees graded the answers like journal papers.
First Proof tests AI on real research problems it cannot have seen before. It is run by mathematicians Mohammed Abouzaid, Nikhil Srivastava, Rachel Ward and Lauren Williams. For the second round (arXiv 2606.18119, 16 June 2026), working mathematicians including Larry Guth, Benny Sudakov and Richard Schwartz contributed ten unpublished problems.
Four systems took part: ProofCouncil, a system from UCLA, ChatGPT 5.5 Pro and Momus. At least two referees graded each of the 39 answers as they would a paper sent to a journal, from “essentially flawless” to “reject”. Seven of the ten problems got at least one passing grade. On problem 4, no system made real progress. Aarhus University reports that the best system, ProofCouncil, solved 6 of the 10.
Costs varied widely, from $117 for ChatGPT 5.5 Pro to $4,799 for the UCLA system. ProofCouncil cost $3,186.
What it means
AI can now produce answers that expert referees accept on some genuinely new research problems, not just exam questions with known answers. It is not reliable yet: three of the ten problems got no passing answer from any system, and even passing answers had citation errors. Every answer still needs an expert to check it.
Caveats
- The 6-of-10 figure comes from Aarhus University’s summary, not directly from the paper’s results table.
- Referees flagged missing or wrong citations even in passing answers.
- Only systems publicly available at the time of the test were included.
- Some Momus runs were spoiled by timeouts.
- Costs are not directly comparable across such different system designs.