Claude 3 Opus

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

1. AgentBench: Evaluating LLMs as Agents — published result table ↗

   Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai et al. · 2023

   All 22 authors

   Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, Jie Tang

2. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? — published result table ↗

   Jimenez, Carlos E., Yang, John, Wettig, Alexander, Yao, Shunyu, Pei, Kexin, Press, Ofir et al. · 2023

   All 7 authors

   Jimenez, Carlos E., Yang, John, Wettig, Alexander, Yao, Shunyu, Pei, Kexin, Press, Ofir, Narasimhan, Karthik

3. WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild — published result table ↗

   Lin, Bill Yuchen, Deng, Yuntian, Chandu, Khyathi, Brahman, Faeze, Ravichander, Abhilasha, Pyatkin, Valentina et al. · 2024

   All 9 authors

   Lin, Bill Yuchen, Deng, Yuntian, Chandu, Khyathi, Brahman, Faeze, Ravichander, Abhilasha, Pyatkin, Valentina, Dziri, Nouha, Bras, Ronan Le, Choi, Yejin

4. MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures — published result table ↗

   Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain et al. · 2024

   All 8 authors

   Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, Yang You

5. $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains — published result table ↗

   Yao, Shunyu, Shinn, Noah, Razavi, Pedram, Narasimhan, Karthik · 2024

6. Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? — published result table ↗

   Lee, Jinhyuk, Chen, Anthony, Dai, Zhuyun, Dua, Dheeru, Sachan, Devendra Singh, Boratko, Michael et al. · 2024

   All 19 authors

   Lee, Jinhyuk, Chen, Anthony, Dai, Zhuyun, Dua, Dheeru, Sachan, Devendra Singh, Boratko, Michael, Luan, Yi, Arnold, Sébastien M. R., Perot, Vincent, Dalmia, Siddharth, Hu, Hexiang, Lin, Xudong, Pasupat, Panupong, Amini, Aida, Cole, Jeremy R., Riedel, Sebastian, Naim, Iftekhar, Chang, Ming-Wei, Guu, Kelvin

7. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents — published result table ↗

   Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, Florian Tramèr · 2024

8. BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions — published result table ↗

   Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari et al. · 2024

   All 33 authors

   Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, Simon Brunner, Chen Gong, Thong Hoang, Armel Randy Zebaze, Xiaoheng Hong, Wen-Ding Li, Jean Kaddour, Ming Xu, Zhihan Zhang, Prateek Yadav, Naman Jain, Alex Gu, Zhoujun Cheng, Jiawei Liu, Qian Liu, Zijian Wang, Binyuan Hui, Niklas Muennighoff, David Lo, Daniel Fried, Xiaoning Du, Harm de Vries, Leandro Von Werra

9. MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data — published result table ↗

   Meng Fang, Xiangpeng Wan, Fei Lu, Fei Xing, Kai Zou · 2024

10. Benchmarking Complex Instruction-Following with Multiple Constraints Composition — published result table ↗

    Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou et al. · 2024

    All 14 authors

    Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, Yiming Liu, Jie Tang, Hongning Wang, Minlie Huang

11. WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queries — published result table ↗

    Wenting Zhao, Tanya Goyal, Yu Ying Chiu, Liwei Jiang, Benjamin Newman, Abhilasha Ravichander et al. · 2024

    All 11 authors

    Wenting Zhao, Tanya Goyal, Yu Ying Chiu, Liwei Jiang, Benjamin Newman, Abhilasha Ravichander, Khyathi Chandu, Ronan Le Bras, Claire Cardie, Yuntian Deng, Yejin Choi

12. SysBench: Can Large Language Models Follow System Messages? — published result table ↗

    Yanzhao Qin, Tao Zhang, Tao Zhang, Yanjun Shen, Wenjing Luo, Haoze Sun et al. · 2024

    All 12 authors

    Yanzhao Qin, Tao Zhang, Tao Zhang, Yanjun Shen, Wenjing Luo, Haoze Sun, Yan Zhang, Yujing Qiao, Weipeng Chen, Zenan Zhou, Wentao Zhang, Bin Cui

13. Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries — published result table ↗

    Vodrahalli, Kiran, Ontanon, Santiago, Tripuraneni, Nilesh, Xu, Kelvin, Jain, Sanil, Shivanna, Rakesh et al. · 2024

    All 24 authors

    Vodrahalli, Kiran, Ontanon, Santiago, Tripuraneni, Nilesh, Xu, Kelvin, Jain, Sanil, Shivanna, Rakesh, Hui, Jeffrey, Dikkala, Nishanth, Kazemi, Mehran, Fatemi, Bahare, Anil, Rohan, Dyer, Ethan, Shakeri, Siamak, Vij, Roopali, Mehta, Harsh, Ramasesh, Vinay, Le, Quoc, Chi, Ed, Lu, Yifeng, Firat, Orhan, Lazaridou, Angeliki, Lespiau, Jean-Baptiste, Attaluri, Nithya, Olszewska, Kate

14. Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models — published result table ↗

    Deitke, Matt, Clark, Christopher, Lee, Sangho, Tripathi, Rohun, Yang, Yue, Park, Jae Sung et al. · 2024

    All 50 authors

    Deitke, Matt, Clark, Christopher, Lee, Sangho, Tripathi, Rohun, Yang, Yue, Park, Jae Sung, Salehi, Mohammadreza, Muennighoff, Niklas, Lo, Kyle, Soldaini, Luca, Lu, Jiasen, Anderson, Taira, Bransom, Erin, Ehsani, Kiana, Ngo, Huong, Chen, YenSung, Patel, Ajay, Yatskar, Mark, Callison-Burch, Chris, Head, Andrew, Hendrix, Rose, Bastani, Favyen, VanderBilt, Eli, Lambert, Nathan, Chou, Yvonne, Chheda, Arnavi, Sparks, Jenna, Skjonsberg, Sam, Schmitz, Michael, Sarnat, Aaron, Bischoff, Byron, Walsh, Pete, Newell, Chris, Wolters, Piper, Gupta, Tanmay, Zeng, Kuo-Hao, Borchardt, Jon, Groeneveld, Dirk, Nam, Crystal, Lebrecht, Sophie, Wittlif, Caitlin, Schoenick, Carissa, Michel, Oscar, Krishna, Ranjay, Weihs, Luca, Smith, Noah A., Hajishirzi, Hannaneh, Girshick, Ross, Farhadi, Ali, Kembhavi, Aniruddha

15. Are large language models superhuman chemists? — published result table ↗

    Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Martiño Ríos-García, Benedict Emoekabu, Aswanth Krishnan et al. · 2024

    All 35 authors

    Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Martiño Ríos-García, Benedict Emoekabu, Aswanth Krishnan, Tanya Gupta, Mara Schilling-Wilhelmi, Macjonathan Okereke, Anagha Aneesh, Amir Mohammad Elahi, Mehrdad Asgari, Juliane Eberhardt, Hani M. Elbeheiry, María Victoria Gil, Maximilian Greiner, Caroline T. Holick, Christina Glaubitz, Tim Hoffmann, Abdelrahman Ibrahim, Lea C. Klepsch, Yannik Köster, Fabian Alexander Kreth, Jakob Meyer, Santiago Miret, Jan Matthias Peschel, Michael Ringleb, Nicole Roesner, Johanna Schreiber, Ulrich S. Schubert, Leanne M. Stafast, Dinga Wonanke, Michael Pieler, Philippe Schwaller, Kevin Maik Jablonka

16. The Claude 3 Model Family: Opus, Sonnet, Haiku ↗

    Anthropic · 2024

17. DTBench — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

18. LMCA — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

19. Holistic Evaluation of Language Models (HELM) ↗

    Stanford Center for Research on Foundation Models

20. Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models ↗

    Zhang, Andy K., Perry, Neil, Dulepet, Riya, Ji, Joey, Menders, Celeste, Lin, Justin W. et al. · 2024

    All 27 authors

    Zhang, Andy K., Perry, Neil, Dulepet, Riya, Ji, Joey, Menders, Celeste, Lin, Justin W., Jones, Eliot, Hussein, Gashon, Liu, Samantha, Jasper, Donovan, Peetathawatchai, Pura, Glenn, Ari, Sivashankar, Vikram, Zamoshchin, Daniel, Glikbarg, Leo, Askaryar, Derek, Yang, Mike, Zhang, Teddy, Alluri, Rishi, Tran, Nathan, Sangpisit, Rinnara, Yiorkadjis, Polycarpos, Osele, Kenny, Raghupathi, Gautham, Boneh, Dan, Ho, Daniel E., Liang, Percy

21. DarkBench: Benchmarking Dark Patterns in Large Language Models — publisher leaderboard data — published result table ↗

    Esben Kran, Hieu Minh Nguyen, Akash Kundu, Sami Jawhar, Jinsuk Park, Mateusz Maria Jurewicz · 2025

22. Chess Puzzles ↗

    Epoch AI

23. GPQA Diamond ↗

    Epoch AI

24. MATH Level 5 ↗

    Epoch AI

25. OTIS Mock AIME 2024-2025 ↗

    Epoch AI

26. SimpleQA Verified ↗

    Epoch AI

27. SimpleBench ↗

    Epoch AI

28. AI benchmarks & capabilities ↗

    Epoch AI

29. WeirdML ↗

    Håvard Tveit Ihle

30. EnigmaEval ↗

    Scale Labs / SEAL

31. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

32. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

33. LiveBench — model evaluation leaderboard ↗

    LiveBench Team

34. Measuring AI Ability to Complete Long Software Tasks ↗

    METR

35. PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change — published result table ↗

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, Subbarao Kambhampati · 2022

36. Simple Evals — model benchmark results ↗

    OpenAI · 2025

37. BLINK: Multimodal Large Language Models Can See but Not Perceive — published result table ↗

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin et al. · 2024

    All 10 authors

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, Ranjay Krishna

38. Grok-1.5 Vision Preview ↗

    xAI · 2024