RLTasksby ulam.ai
Models / Anthropic

Anthropic / MODEL EVIDENCE

Claude Sonnet 4

45 benchmarks · 88 observations · 47 sources · released 2025-05-22

Scores at a glance

45 benchmarks · large dot: Claude Sonnet 4; small dots: closest models where reported

GPQA Diamond ↗79.19%
79.1979.19

Claude Sonnet 4 · GPQA diamond · Best score (across scorers) · % ↗

5 more reported results

78.28% · Claude Sonnet 4 · GPQA diamond · Best score (across scorers) · % · Epoch AI · GPQA diamond ↗

75.76% · Claude Sonnet 4 · GPQA diamond · Best score (across scorers) · % · Epoch AI · GPQA diamond ↗

66.67% · Claude Sonnet 4 · GPQA diamond · Best score (across scorers) · % · Epoch AI · GPQA diamond ↗

75 % · Claude Sonnet 4 (Thinking) · GPQA Diamond · Accuracy · Vals AI · GPQA Diamond ↗

69.44 % · Claude Sonnet 4 (Nonthinking) · GPQA Diamond · Accuracy · Vals AI · GPQA Diamond ↗

MATH Level 5 ↗84.37%
3.2998.13

Claude Sonnet 4 · MATH level 5 · Best score (across scorers) · % ↗

OTIS Mock AIME 2024-2025 ↗71.11%
71.1171.11

Claude Sonnet 4 · OTIS Mock AIME 2024-2025 · Best score (across scorers) · % ↗

3 more reported results

68.89% · Claude Sonnet 4 · OTIS Mock AIME 2024-2025 · Best score (across scorers) · % · Epoch AI · OTIS Mock AIME 2024-2025 ↗

53.33% · Claude Sonnet 4 · OTIS Mock AIME 2024-2025 · Best score (across scorers) · % · Epoch AI · OTIS Mock AIME 2024-2025 ↗

28.89% · Claude Sonnet 4 · OTIS Mock AIME 2024-2025 · Best score (across scorers) · % · Epoch AI · OTIS Mock AIME 2024-2025 ↗

Cybench ↗35%
555

Claude Sonnet 4 · Cybench · Unguided % Solved · % ↗

GSO ↗4.9%
026.5

Claude Sonnet 4 · GSO-Bench · Score OPT@1 · % ↗

1 more reported result

4.9% · Claude Sonnet 4 (unknown thinking) · GSO-Bench · Score OPT@1 · % · Epoch AI · GSO-Bench ↗

CritPt ↗0.29%
032.29

Claude Sonnet 4 (unknown thinking) · CritPt · Accuracy · % ↗

LMCA ↗29%
2.7868.26

Claude Sonnet 4 (unknown thinking) · LMCA · Score · % ↗

DTBench ↗77.07%
41.6598.93

Claude Sonnet 4 (unknown thinking) · DTBench · Accuracy · % ↗

SciCode ↗40.05%
1.2766.9

Claude Sonnet 4 (unknown thinking) · SciCode · Score · % ↗

MindCube ↗44.75%
29.4646.67

Claude Sonnet 4 (unknown thinking) · MindCube · Overall score · % ↗

EnigmaEval ↗3.12%
0.3839.28

Claude Sonnet 4 (unknown thinking) · EnigmaEval · Accuracy · % ↗

MedCode ↗34.96 %
34.9634.96

Claude Sonnet 4 (Thinking) · MedCode · Accuracy ↗

1 more reported result

33.94 % · Claude Sonnet 4 (Nonthinking) · MedCode · Accuracy · Vals AI · MedCode ↗

MedScribe ↗72.41 %
72.4172.41

Claude Sonnet 4 (Nonthinking) · MedScribe · Accuracy ↗

1 more reported result

69.35 % · Claude Sonnet 4 (Thinking) · MedScribe · Accuracy · Vals AI · MedScribe ↗

MATH-500 ↗93.8 %
93.893.8

Claude Sonnet 4 (Thinking) · MATH 500 · Accuracy ↗

1 more reported result

90.32 % · Claude Sonnet 4 (Nonthinking) · MATH 500 · Accuracy · Vals AI · MATH 500 ↗

MMLU-Pro ↗83.86 %
83.8683.86

Claude Sonnet 4 (Thinking) · MMLU Pro · Accuracy ↗

1 more reported result

79.43 % · Claude Sonnet 4 (Nonthinking) · MMLU Pro · Accuracy · Vals AI · MMLU Pro ↗

MMMU-Pro ↗74.93 %
74.9374.93

Claude Sonnet 4 (Thinking) · MMMU Pro · Accuracy ↗

1 more reported result

72.39 % · Claude Sonnet 4 (Nonthinking) · MMMU Pro · Accuracy · Vals AI · MMMU Pro ↗

Fortress ↗24.37%
5.7874.39

Claude Sonnet 4 · Scale Labs · Risk elicitation score ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

10.1 percentile-point gap

6 of 6 shared benchmark comparisons

20.5 percentile-point gap

6 of 6 shared benchmark comparisons

21.5 percentile-point gap

4 of 6 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. Aider LLM Leaderboards — Polyglot coding benchmark ↗

    Aider

  2. ARC Prize - Leaderboard ↗

    ARC Prize Foundation

  3. SciCode Benchmark | Artificial Analysis ↗

    Artificial Analysis

  4. TextQuests: How Good are LLMs at Text-Based Video Games? — published result table ↗

    Phan, Long, Mazeika, Mantas, Zou, Andy, Hendrycks, Dan · 2025

  5. DTBench — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

  6. LMCA — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

  7. Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models ↗

    Zhang, Andy K., Perry, Neil, Dulepet, Riya, Ji, Joey, Menders, Celeste, Lin, Justin W. et al. · 2024

    All 27 authors

    Zhang, Andy K., Perry, Neil, Dulepet, Riya, Ji, Joey, Menders, Celeste, Lin, Justin W., Jones, Eliot, Hussein, Gashon, Liu, Samantha, Jasper, Donovan, Peetathawatchai, Pura, Glenn, Ari, Sivashankar, Vikram, Zamoshchin, Daniel, Glikbarg, Leo, Askaryar, Derek, Yang, Mike, Zhang, Teddy, Alluri, Rishi, Tran, Nathan, Sangpisit, Rinnara, Yiorkadjis, Polycarpos, Osele, Kenny, Raghupathi, Gautham, Boneh, Dan, Ho, Daniel E., Liang, Percy

  8. FutureSearch Evals ↗

    FutureSearch

  9. ARC-AGI-2 ↗

    Epoch AI

  10. CritPt v1 ↗

    Epoch AI

  11. FrontierMath Tier 4 (v1, old) ↗

    Epoch AI

  12. FrontierMath Tiers 1-3 (v1, old) ↗

    Epoch AI

  13. GPQA Diamond ↗

    Epoch AI

  14. Humanity's Last Exam ↗

    Epoch AI

  15. MATH Level 5 ↗

    Epoch AI

  16. OTIS Mock AIME 2024-2025 ↗

    Epoch AI

  17. SimpleBench ↗

    Epoch AI

  18. The Agent Company ↗

    Epoch AI

  19. Capabilities & benchmarking — retained external evaluation records ↗

    Epoch AI · 2026

  20. AI benchmarks & capabilities ↗

    Epoch AI

  21. Fiction.liveBench — March 14, 2025 benchmark release ↗

    Fiction.live · 2025

  22. GeoBench — GeoGuessr model benchmark ↗

    GeoBench

  23. Writing benchmark — incorporating mandatory story elements ↗

    lechmazur

  24. GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents ↗

    Shetty, Manish, Jain, Naman, Liu, Jinjian, Kethanaboyina, Vijay, Sen, Koushik, Stoica, Ion · 2025

  25. WeirdML ↗

    Håvard Tveit Ihle

  26. EnigmaEval ↗

    Scale Labs / SEAL

  27. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  28. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  29. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  30. Measuring AI Ability to Complete Long Software Tasks ↗

    METR

  31. MindCube: Spatial Mental Modeling from Limited Views ↗

    Qineng Wang, Baiqiao Yin, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang et al.

    All 14 authors

    Qineng Wang, Baiqiao Yin, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Jiajun Wu, Li Fei-Fei, Manling Li

  32. OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments ↗

    Xie, Tianbao, Zhang, Danyang, Chen, Jixuan, Li, Xiaochuan, Zhao, Siheng, Cao, Ruisheng et al. · 2024

    All 17 authors

    Xie, Tianbao, Zhang, Danyang, Chen, Jixuan, Li, Xiaochuan, Zhao, Siheng, Cao, Ruisheng, Hua, Toh Jing, Cheng, Zhoujun, Shin, Dongchan, Lei, Fangyu, Liu, Yitao, Xu, Yiheng, Zhou, Shuyan, Savarese, Silvio, Xiong, Caiming, Zhong, Victor, Yu, Tao

  33. AIME — evaluation methodology and results ↗

    Vals AI · 2026

  34. CorpFin — evaluation methodology and results ↗

    Vals AI · 2026

  35. GPQA Diamond — evaluation methodology and results ↗

    Vals AI · 2026

  36. LiveCodeBench — evaluation methodology and results ↗

    Vals AI · 2026

  37. LegalBench — evaluation methodology and results ↗

    Vals AI · 2026

  38. MATH 500 — evaluation methodology and results ↗

    Vals AI · 2026

  39. MedCode — evaluation methodology and results ↗

    Vals AI · 2026

  40. MedQA — evaluation methodology and results ↗

    Vals AI · 2026

  41. MedScribe — evaluation methodology and results ↗

    Vals AI · 2026

  42. MGSM — evaluation methodology and results ↗

    Vals AI · 2026

  43. MMLU Pro — evaluation methodology and results ↗

    Vals AI · 2026

  44. MMMU Pro — evaluation methodology and results ↗

    Vals AI · 2026

  45. MortgageTax — evaluation methodology and results ↗

    Vals AI · 2026

  46. SAGE — evaluation methodology and results ↗

    Vals AI · 2026

  47. TaxEval — evaluation methodology and results ↗

    Vals AI · 2026

Names used in source tables

Claude Sonnet 4 · Claude Sonnet 4 (Nonthinking) · Claude Sonnet 4 (Thinking) · Claude Sonnet 4 (unknown thinking)