Myth check: “AI can't create new knowledge, it only remixes its training data”
AI disproved an 80-year-old conjecture and designed antibiotics that worked in mice. Real finds come where answers can be checked.
The claim in full “AI only recombines what it was trained on, so it can't discover anything genuinely new.”
Open research problems solved by AI (FrontierMath Erdős)
Share of 68 significant open Erdős problems a model solves with a machine-checked proof, in one attempt with a ~$300 budget.
What this measures: Unlike exam-style tests, nobody knew the answers to these problems when the test was made. A model only scores if the Lean proof checker accepts its proof or disproof.
Plotted at model release dates (the benchmark launched 1 September 2026, so earlier models were evaluated retroactively). Outside the official one-attempt rules, 17 of the 68 problems have now been solved by some means, including 12 credited to OpenAI's 6 October research release. Contamination risk grows as solutions are published.
Show data table and sources
| Date | Label | Value |
|---|---|---|
| 23 Apr 2026 | GPT-5.5 | 0% |
| 3 Sept 2026 | GPT-6 Astra (pre-release) | 2.9% |
| 29 Sept 2026 | GPT-6.1 Sol | 4.4% |
- Announcing FrontierMath Erdős (Epoch AI)
- FrontierMath Erdős benchmark page (Epoch AI)
- Epoch AI benchmark data (CSV export) (Epoch AI)
What the data shows
The argument: a language model learns to predict text that people wrote, so all it can do is remix what people already knew. It can’t find out something nobody knew.
What’s true. A lot of what has been sold as AI discovery wasn’t. In October 2025 an OpenAI executive said GPT-5 had solved 10 open Erdős problems (puzzles posed by the mathematician Paul Erdős). It had found answers already published. On Epoch AI’s test of 68 significant open Erdős problems, the best model solves 3 under strict rules.
What the data shows. Since 2025, AI systems have produced results nobody had found before, checked by outside experts:
- In May 2026 an unreleased OpenAI model found a counterexample to Erdős’s 1946 unit distance conjecture (a statement mathematicians believed but had not proved). Nine mathematicians, including Tim Gowers and Noga Alon, checked and published it the same day.
- Google DeepMind’s AlphaEvolve found a way to multiply 4×4 grids of complex numbers (matrices) with 48 multiplications, beating a record of 49 that had stood for 56 years.
- MIT researchers used AI to design new antibiotic molecules. Of 24 that chemists could make, 7 killed bacteria and 2 cleared drug-resistant infections in mice (Cell, August 2025).
- OpenAI’s Astra model produced ten results in maths and theoretical computer science, each with a computer-checked proof that anyone can re-run.
What it means
AI can find things nobody knew, so far mainly where an answer can be checked: proofs, algorithms and molecules tested in a lab. That is also where overclaims get caught quickly. Trust the results checked by outside experts or by a computer, and treat a lab’s announcement on its own with care.
Caveats
- Most of these results came from systems built around a model (search, automatic checkers, thousands of attempts), not a chatbot answering once.
- Labs report successes and rarely report failures or total cost.
- Remixing and discovery aren’t cleanly separable: much human discovery also combines known ideas in a new way.
- None of the AI-designed antibiotics from the MIT work has been tested in people.