GPT-J-6B · ARC AI2 · Challenge score · % ↗
OpenAI / MODEL EVIDENCE
GPT-J-6B
11 benchmarks · 15 observations · 6 sources · released 2021-08-05
Scores at a glance
11 benchmarks · large dot: GPT-J-6B; small dots: closest models where reported
GPT-J (6B) · GSM8K · EM · % ↗
GPT-J (6B) · HellaSwag · Overall accuracy · % ↗
1 more reported result
66.2% · GPT-J-6B · HellaSwag · Overall accuracy · % · Epoch AI · HellaSwag ↗
GPT-J-6B · MMLU · EM · % ↗
1 more reported result
25.1% · GPT-J-6B · MMLU · EM · % · Epoch AI · MMLU ↗
GPT-J-6B · OpenBookQA · Accuracy · % ↗
GPT-J-6B · PIQA · Score · % ↗
GPT-J-6B · Winogrande · Accuracy · % ↗
GPT-J-6B · HumanEval+ · Pass@1 · EvalPlus · official result data ↗
GPT-J-6B · HumanEval · Pass@1 · EvalPlus · official result data ↗
GPT-J 6B · TruthfulQA · Human-rated truthfulness · Official benchmark repository · published comparison ↗
1 more reported result
36% · GPT-J 6B · TruthfulQA · Multiple-choice MC2 · Official benchmark repository · published comparison · TruthfulQA: Measuring How Models Mimic Human Falsehoods ↗
GPT-J (6B) · Accuracy ↗
1 more reported result
65.4% · GPT-J-6B · Accuracy · Epoch AI · BoolQ · public result data ↗
Closest models by score
Similar positions on shared benchmark leaderboards. Smaller gaps are closer.
There are not enough shared scored settings to identify close comparisons reliably.
Sources
Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.
- XGen-7B Technical Report ↗
All 25 authors
Nijkamp, Erik, Xie, Tian, Hayashi, Hiroaki, Pang, Bo, Xia, Congying, Xing, Chen, Vig, Jesse, Yavuz, Semih, Laban, Philippe, Krause, Ben, Purushwalkam, Senthil, Niu, Tong, Kryściński, Wojciech, Murakhovs'ka, Lidiya, Choubey, Prafulla Kumar, Fabbri, Alex, Liu, Ye, Meng, Rui, Tu, Lifu, Bhat, Meghana, Wu, Chien-Sheng, Savarese, Silvio, Zhou, Yingbo, Joty, Shafiq, Xiong, Caiming
- Holistic Evaluation of Language Models (HELM) ↗
- Holistic Evaluation of Language Models (HELM) ↗
- Holistic Evaluation of Language Models (HELM) ↗
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation — published result table ↗
- TruthfulQA: Measuring How Models Mimic Human Falsehoods — published result table ↗
Names used in source tables
GPT-J (6B) · GPT-J 6B · GPT-J-6B