Skip to content

Anthropic

Claude 3.5 Sonnet (Oct 2024)

Announced
Access
Public

Benchmark scores

TestScoreBest on the trackerHow it was measured
Humanity's Last Examexpert questions, no tools4.1%GPT-6 Astra 55%Independentleaderboard entry "Claude 3.5 Sonnet (October 2024)" (claude-3-5-sonnet-20241022)
GPQA DiamondPhD-level science questionsPhD experts: 70%Saturated55%Gemini 4 Argon 96%Independentdefault settings (claude-3-5-sonnet-20241022)Run 27 Jan 2025
OTIS Mock AIMEcompetition mathsSaturated8.5%GPT-5.5 100%Independentdefault settings (claude-3-5-sonnet-20241022)Run 25 Feb 2025
METR time horizontask length an AI agent finishes half the timeNo longer updated21 minClaude Mythos Preview 17 hIndependentMETR Horizon v1.1 run 'claude_3_5_sonnet_20241022_inspect'; 95% CI 10.1–40.8 min

Compare every model on the same tests →

Also on these charts

Sources