o3 beat 34 of 36 virologists on a written lab test, but AI gave novices no clear lab boost
On written lab-problem questions, o3 scored 43.8% to virologists' 22.1%. In a 2026 trial, AI did not significantly help novices in the lab.
The Virology Capabilities Test, from SecureBio, the Center for AI Safety and MIT, asks hard questions about fixing problems in virology lab work. The concern behind it is biosecurity: whether AI could help someone work with dangerous viruses.
On it, OpenAI’s o3 scored 43.8%. Thirty-six PhD virologists averaged 22.1% on questions in their own specialties, and o3 beat 34 of them. Gemini 2.5 Pro scored 37.6% and GPT-4o 18.8%. The questions are kept private so they don’t leak into training data.
Labs now use results like this in safety decisions. Anthropic said rising scores on tests like this were part of why it switched on stricter safeguards in May 2025.
Written knowledge isn’t lab skill, though. A pre-registered randomised trial (its plan published in advance, people assigned to groups at random), posted in February 2026, followed 153 novices in a teaching lab for eight weeks. It reportedly found that access to AI models did not significantly improve how well they completed the full lab workflow.
What it means
On paper, AI now knows more about troubleshooting virology lab work than most specialists, which is one reason labs have added safeguards. The one trial so far found this did not turn into practical help for beginners at the bench. That is a single study, reported second-hand, so the question stays open.
Caveats
- The test measures written troubleshooting knowledge under all-or-nothing scoring, not hands-on skill.
- The question set is private, so outside groups can’t reproduce the scores.
- The lab-trial details come from a review of the preprint, not the preprint itself.