Microsoft researchers hid a trigger in an AI agent by editing 7 of its neurons, without retraining
The edited model still passed maths and coding tests, and a trigger made it emit a planted command. A checker caught 91% of edited models.
Researchers at Microsoft Research and the Microsoft AI Red Team showed that an AI agent’s weights can be edited by hand to hide a “sleeper” behaviour. Their method, Lazarus, takes dormant neurons in the model’s last feed-forward layer, ones that barely affect its output, and rewrites them so that a trigger string in the agent’s context makes the model write a chosen command. The edit is made without training the model, and the paper says a command of P tokens needs about P edited neurons.
On NVIDIA’s Nemotron-Terminal-8B, seven neurons, tuned on 23 example runs, produced the planted command in at least 98.8% of triggered test runs and in none of the untriggered ones. On Qwen3.6-27B, six neurons gave a 51% success rate with 23 examples and 83.7% with about 2,400.
The edit changed ordinary performance only slightly. On Qwen3.6-27B, AIME 2025 (30 maths problems, one attempt each) fell from 61.7% to between 57.5% and 60.4%, and Terminal-Bench 2.1 (88 command-line tasks) from 46.6% to 42.0%.
The authors also built defences. Two frontier models, each given one edited or clean model and no original to compare against, were asked to audit it. GPT-6 Astra called 72.8% of 114 edited models tampered. Claude Opus 5.5 caught 56.1% and called 41.2% clean. A checker that looks for neurons with unusually large weights flagged 91.4% of edited test models and wrongly flagged 9.7% of unedited ones. The authors recommend that model hubs compare uploads with their declared base model and audit weights, and that users limit what agents can access.
What it means
Anyone who downloads a modified copy of an open model, such as a quantised or fine-tuned version from a public hub, cannot learn from benchmark scores alone whether it hides a trigger. The harm is capped by what the agent is allowed to do, so an agent with shell and network access is the risky case. Nothing in the paper shows this attack used outside the lab.
Caveats
- This is a defenders’ research demonstration by Microsoft’s own researchers. No attack in the wild is reported, and the figures are self-reported.
- “Success” means the model predicts the planted command in recorded runs, not that an agent was seen carrying out an attack from start to finish. The authors list full end-to-end runs as future work.
- Attack success varies a lot by model and setting: Gemma-4-31B-it reached 62.2% in one 8-step test, and a single Qwen edit reached 40% on an agent format it had not seen.
- The 91.4% detector was tested on the same kind of edit it was built for, so it may miss other methods. The benchmark tests were small, and Terminal-Bench fell 4.6 points.
- The blog post gives no numbers. All figures here come from the paper, which is a PDF hosted on the lead researcher’s GitHub; we found no peer review.