RLTasksby ulam.ai
Models / Meta

Meta / MODEL EVIDENCE

Llama 3-8B-Instruct

13 benchmarks · 47 observations · 11 sources

Scores at a glance

13 benchmarks · large dot: Llama 3-8B-Instruct; small dots: closest models where reported

RepoQA & LongCodeArena ↗53.6%
2.290.6

Meta-Llama-3-8B-Instruct · RepoQA & LongCodeArena · RepoQA retrieval accuracy · similarity threshold 0.8 ↗

ComplexBench ↗0.638%
0.550.8

Llama-3-8B-Instruct · ComplexBench · DRFR · all constraints · RAL evaluation ↗

SORRY-Bench ↗22%
1668

Llama-3-8b-instruct · SORRY-Bench · Base harmful compliance · original prompts ↗

HumanEval+ ↗56.7%
1.889

Llama3-8B-instruct · HumanEval+ · Pass@1 · EvalPlus · official result data ↗

MBPP+ ↗54.8%
7.580.2

Llama3-8B-instruct · MBPP+ · Pass@1 · EvalPlus · official result data ↗

HumanEval ↗61.6%
2.496.3

Llama3-8B-instruct · HumanEval · Pass@1 · EvalPlus · official result data ↗

Putnam-AXIOM ↗5.75%
0.9541.94

Llama-3-8b Instruct · Putnam-AXIOM · Original · problem accuracy · original questions ↗

ProcessBench ↗11.1 F1 points
5.187.9

Meta-Llama-3-8B-Instruct · ProcessBench · Average F1 · original comparison ↗

CFBench (Chinese Complex Following) ↗60.9%
60.990.8

Llama-3-8B-Instruct∗ · CFBench (Chinese Complex Following) · Full set · Constraint satisfaction · full-set evaluation ↗

2 more reported results

21.1% · Llama-3-8B-Instruct∗ · CFBench (Chinese Complex Following) · Full set · Instruction satisfaction · full-set evaluation · CFBench: A Comprehensive Constraints-Following Benchmark for LLMs ↗

29.7% · Llama-3-8B-Instruct∗ · CFBench (Chinese Complex Following) · Full set · Priority satisfaction · full-set evaluation · CFBench: A Comprehensive Constraints-Following Benchmark for LLMs ↗

BABILong ↗64%
2799.36

Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1–QA5 average · 0k context · historical v0 leaderboard ↗

29 more reported results

50% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1–QA5 average · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

60% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1–QA5 average · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

58% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1–QA5 average · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

44% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1–QA5 average · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

98% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

93% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

90% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

79% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

62% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA1 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

47% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA2 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

46% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA2 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

49% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA2 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

39% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA2 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

20% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA2 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

33% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA3 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

28% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA3 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

30% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA3 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

26% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA3 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

11% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA3 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

58% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA4 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

55% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA4 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

50% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA4 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

43% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA4 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

52% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA4 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

85% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA5 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

78% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA5 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

73% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA5 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

65% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA5 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

73% · Meta-Llama-3-8B-Instruct · BABILong · BABILong v0 · QA5 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

There are not enough shared scored settings to identify close comparisons reliably.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. RepoQA: Evaluating Long Context Code Understanding — published result table ↗

    Jiawei Liu, Jia Le Tian, Vijay Daita, Yuxiang Wei, Yifeng Ding, Yuhan Katherine Wang et al. · 2024

    All 8 authors

    Jiawei Liu, Jia Le Tian, Vijay Daita, Yuxiang Wei, Yifeng Ding, Yuhan Katherine Wang, Jun Yang, Lingming Zhang

  2. MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures — published result table ↗

    Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain et al. · 2024

    All 8 authors

    Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, Yang You

  3. SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal — published result table ↗

    Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang et al. · 2024

    All 16 authors

    Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, Prateek Mittal

  4. BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions — published result table ↗

    Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari et al. · 2024

    All 33 authors

    Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, Simon Brunner, Chen Gong, Thong Hoang, Armel Randy Zebaze, Xiaoheng Hong, Wen-Ding Li, Jean Kaddour, Ming Xu, Zhihan Zhang, Prateek Yadav, Naman Jain, Alex Gu, Zhoujun Cheng, Jiawei Liu, Qian Liu, Zijian Wang, Binyuan Hui, Niklas Muennighoff, David Lo, Daniel Fried, Xiaoning Du, Harm de Vries, Leandro Von Werra

  5. Benchmarking Complex Instruction-Following with Multiple Constraints Composition — published result table ↗

    Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou et al. · 2024

    All 14 authors

    Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, Yiming Liu, Jie Tang, Hongning Wang, Minlie Huang

  6. CFBench: A Comprehensive Constraints-Following Benchmark for LLMs — published result table ↗

    Tao Zhang, Chenglin Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang et al. · 2024

    All 14 authors

    Tao Zhang, Chenglin Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang, Zenan Zhou

  7. ProcessBench: Identifying Process Errors in Mathematical Reasoning — published result table ↗

    Zheng, Chujie, Zhang, Zhenru, Zhang, Beichen, Lin, Runji, Lu, Keming, Yu, Bowen et al. · 2024

    All 9 authors

    Zheng, Chujie, Zhang, Zhenru, Zhang, Beichen, Lin, Runji, Lu, Keming, Yu, Bowen, Liu, Dayiheng, Zhou, Jingren, Lin, Junyang

  8. Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs — published result table ↗

    Gulati, Aryan, Miranda, Brando, Chen, Eric, Xia, Emily, Fronsdal, Kai, Dumont, Bruno et al. · 2025

    All 8 authors

    Gulati, Aryan, Miranda, Brando, Chen, Eric, Xia, Emily, Fronsdal, Kai, Dumont, Bruno, Obbad, Elyas, Koyejo, Sanmi

  9. Are large language models superhuman chemists? — published result table ↗

    Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Martiño Ríos-García, Benedict Emoekabu, Aswanth Krishnan et al. · 2024

    All 35 authors

    Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Martiño Ríos-García, Benedict Emoekabu, Aswanth Krishnan, Tanya Gupta, Mara Schilling-Wilhelmi, Macjonathan Okereke, Anagha Aneesh, Amir Mohammad Elahi, Mehrdad Asgari, Juliane Eberhardt, Hani M. Elbeheiry, María Victoria Gil, Maximilian Greiner, Caroline T. Holick, Christina Glaubitz, Tim Hoffmann, Abdelrahman Ibrahim, Lea C. Klepsch, Yannik Köster, Fabian Alexander Kreth, Jakob Meyer, Santiago Miret, Jan Matthias Peschel, Michael Ringleb, Nicole Roesner, Johanna Schreiber, Ulrich S. Schubert, Leanne M. Stafast, Dinga Wonanke, Michael Pieler, Philippe Schwaller, Kevin Maik Jablonka

  10. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation — published result table ↗

    Liu, Jiawei, Xia, Chunqiu Steven, Wang, Yuyao, Zhang, Lingming · 2023

  11. BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard — published result table ↗

    Kuratov, Yuri, Bulatov, Aydar, Anokhin, Petr, Rodkin, Ivan, Sorokin, Dmitry, Sorokin, Artyom et al. · 2024

    All 7 authors

    Kuratov, Yuri, Bulatov, Aydar, Anokhin, Petr, Rodkin, Ivan, Sorokin, Dmitry, Sorokin, Artyom, Burtsev, Mikhail

Names used in source tables

LLaMA-3-8B-Instruct · Llama-3-8B-Instruct · Llama-3-8B-Instruct∗ · Llama-3-8b Instruct · Llama-3-8b-instruct · Llama3-8B-instruct · Llama3-instruct / 8B · Meta-Llama-3-8B-Instruct