AI rose from 31% to 83.5% on fixing real code bugs, then OpenAI dropped the test as flawed
Built from real bug reports, SWE-bench Verified was a standard coding test. OpenAI dropped it in 2026 over broken tests and memorised fixes.
31% → 83.5%best independent SWE-bench Verified score, Nov 2024 → Apr 2026