TriviaQA

An open-domain question answering benchmark with challenging trivia questions paired with evidence documents.26 Oct 2026

One row per model and compatible score measure. Mean = equal-weight arithmetic average of imported reports. Distinct configurations reported for the same model in a source table remain separate. Reports can overlap; the range is not a confidence interval. Unbundle to inspect each observation.
		Evidence	What the result shows
Claude 2 →Anthropic	87.5% 1 report	TriviaQA[1]claude-2.0Released 2023-07-11	Leads this displayed comparison.
Claude 1.3 →Anthropic	86.7% 1 report	TriviaQA[1]claude-1.3Released 2023-04-18	#2 of 40 in this displayed comparison; 0.8 pp below the leader.
PaLM 2-L →	86.1% 1 report	TriviaQA[3]PaLM 2-LReleased 2023-05-17	#3 of 40 in this displayed comparison; 1.4 pp below the leader.
GPT-3.5 Turbo (Nov 2023) →OpenAI	85.8% 1 report	TriviaQA[9]gpt-3.5-turbo-1106Released 2023-11-06	#4 of 40 in this displayed comparison; 1.7 pp below the leader.
Llama 2-70B →Meta	85.5% 5 reports · range 82.4–87.6%	TriviaQA[12][4] View 5 reports 87.6% · TriviaQA · EM · % · Llama-2-70b-hf [12] 87.5% · TriviaQA · EM · % · Llama-2-70b-hf [12] 85% · TriviaQA · EM · % · Llama-2-70b-hf [4] 85% · TriviaQA · EM · % · Llama-2-70b-hf [12] 82.4% · TriviaQA · EM · % · Llama-2-70b-hf [12]	Published observations span 5.2 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #5 of 40 in this displayed comparison; 2 pp below the leader.
GPT-4 (Jun 2023) →OpenAI	84.8% 1 report	TriviaQA[2]gpt-4-0613Released 2023-06-13	#6 of 40 in this displayed comparison; 2.7 pp below the leader.
Llama 2-34B →Meta	83.35% 4 reports · range 81–84.6%	TriviaQA[12] View 4 reports 84.6% · TriviaQA · EM · % · Llama-2-34b [12] 84.5% · TriviaQA · EM · % · Llama-2-34b [12] 83.3% · TriviaQA · EM · % · Llama-2-34b [12] 81% · TriviaQA · EM · % · Llama-2-34b [12]	Published observations span 3.6 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #7 of 40 in this displayed comparison; 4.15 pp below the leader.
DeepSeek V3 →DeepSeek	82.9% 1 report	TriviaQA[8]DeepSeek-V3Released 2024-12-26	#8 of 40 in this displayed comparison; 4.6 pp below the leader.
Llama 3.1-405B →Meta	82.7% 1 report	TriviaQA[8]Llama-3.1-405BReleased 2024-07-23	#9 of 40 in this displayed comparison; 4.8 pp below the leader.
Mixtral 8x7B →Mistral	82.2% 1 report	TriviaQA[9]Mixtral-8x7B-v0.1Released 2023-12-11	#10 of 40 in this displayed comparison; 5.3 pp below the leader.
PaLM 2-M →	81.7% 1 report	TriviaQA[3]PaLM 2-MReleased 2023-05-17	#11 of 40 in this displayed comparison; 5.8 pp below the leader.
DeepSeek V2 (MoE-236B, May 2024) →DeepSeek	80% 1 report	TriviaQA[8]DeepSeek-V2Released 2024-05-07	#12 of 40 in this displayed comparison; 7.5 pp below the leader.
PaLM (540B) →Google	79.9% 3 reports · range 76.9–81.4%	TriviaQA[5] View 3 reports 81.4% · TriviaQA · EM · % · PaLM 540B [5] 81.4% · TriviaQA · EM · % · PaLM 540B [5] 76.9% · TriviaQA · EM · % · PaLM 540B [5]	Published observations span 4.5 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #13 of 40 in this displayed comparison; 7.6 pp below the leader.
Claude Instant →Anthropic	78.9% 1 report	TriviaQA[1]claude-instant-1.1Released 2023-08-09	#14 of 40 in this displayed comparison; 8.6 pp below the leader.
Claude Instant →Anthropicclaude-instant-1.1	78.9% 1 report	TriviaQA[15]claude-instant-1.1Released 2023-08-09	#14 of 40 in this displayed comparison; 8.6 pp below the leader.
Claude Instant →Anthropicclaude-instant-1.2	78.7% 1 report	TriviaQA[15]claude-instant-1.2Released 2023-08-09	#16 of 40 in this displayed comparison; 8.8 pp below the leader.
Falcon-40B →Technology Innovation Institute	78.17% 4 reports · range 74.6–79.9%	TriviaQA[12] View 4 reports 79.9% · TriviaQA · EM · % · falcon-40b [12] 79.6% · TriviaQA · EM · % · falcon-40b [12] 78.6% · TriviaQA · EM · % · falcon-40b [12] 74.6% · TriviaQA · EM · % · falcon-40b [12]	Published observations span 5.3 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #17 of 40 in this displayed comparison; 9.33 pp below the leader.
LLaMA-65B →Meta	77.94% 8 reports · range 68.2–86%	TriviaQA[12][13] View 8 reports 86% · TriviaQA · EM · % · LLaMA-65B [12] 85.9% · TriviaQA · EM · % · LLaMA-65B [12] 84.5% · TriviaQA · EM · % · LLaMA-65B [12] 81.7% · TriviaQA · EM · % · LLaMA-65B [12] 73% · TriviaQA · EM · % · LLaMA-65B [13] 72.6% · TriviaQA · EM · % · LLaMA-65B [13] 71.6% · TriviaQA · EM · % · LLaMA-65B [13] 68.2% · TriviaQA · EM · % · LLaMA-65B [13]	Published observations span 17.8 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #18 of 40 in this displayed comparison; 9.56 pp below the leader.
Llama 2-13B →Meta	77.78% 5 reports · range 73.1–79.6%	TriviaQA[7][12] View 5 reports 79.6% · TriviaQA · EM · % · Llama-2-13b [7] 79.6% · TriviaQA · EM · % · Llama-2-13b [12] 79.4% · TriviaQA · EM · % · Llama-2-13b [12] 77.2% · TriviaQA · EM · % · Llama-2-13b [12] 73.1% · TriviaQA · EM · % · Llama-2-13b [12]	Published observations span 6.5 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #19 of 40 in this displayed comparison; 9.72 pp below the leader.
PaLM 2-S →	75.2% 1 report	TriviaQA[3]PaLM 2-SReleased 2023-05-17	#20 of 40 in this displayed comparison; 12.3 pp below the leader.
LLaMA-33B →Meta	75.01% 8 reports · range 65.1–83.8%	TriviaQA[12][13] View 8 reports 83.8% · TriviaQA · EM · % · LLaMA-33B [12] 83.6% · TriviaQA · EM · % · LLaMA-33B [12] 80.7% · TriviaQA · EM · % · LLaMA-33B [12] 78.7% · TriviaQA · EM · % · LLaMA-33B [12] 70.4% · TriviaQA · EM · % · LLaMA-33B [13] 69.9% · TriviaQA · EM · % · LLaMA-33B [13] 67.9% · TriviaQA · EM · % · LLaMA-33B [13] 65.1% · TriviaQA · EM · % · LLaMA-33B [13]	Published observations span 18.7 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #21 of 40 in this displayed comparison; 12.49 pp below the leader.
GLaM →Google	74.3% 3 reports · range 71.3–75.8%	TriviaQA[10] View 3 reports 75.8% · TriviaQA · EM · % · GLaM (MoE) [10] 75.8% · TriviaQA · EM · % · GLaM (MoE) [10] 71.3% · TriviaQA · EM · % · GLaM (MoE) [10]	Published observations span 4.5 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #22 of 40 in this displayed comparison; 13.2 pp below the leader.
phi-3-medium 14B →Microsoft	73.9% 1 report	TriviaQA[9]Phi-3-medium-128k-instructReleased 2024-04-23	#23 of 40 in this displayed comparison; 13.6 pp below the leader.
Qwen2.5-72B →Qwen	71.9% 1 report	TriviaQA[8]Qwen2.5-72BReleased 2024-09-19	#24 of 40 in this displayed comparison; 15.6 pp below the leader.
MPT-30B →MosaicML	71.55% 4 reports · range 68–73.6%	TriviaQA[12] View 4 reports 73.6% · TriviaQA · EM · % · mpt-30b [12] 73.3% · TriviaQA · EM · % · mpt-30b [12] 71.3% · TriviaQA · EM · % · mpt-30b [12] 68% · TriviaQA · EM · % · mpt-30b [12]	Published observations span 5.6 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #25 of 40 in this displayed comparison; 15.95 pp below the leader.
Llama 2-7B →Meta	70.52% 5 reports · range 65.8–73.7%	TriviaQA[12][7] View 5 reports 73.7% · TriviaQA · EM · % · Llama-2-7b [12] 72.1% · TriviaQA · EM · % · Llama-2-7b [7] 72.1% · TriviaQA · EM · % · Llama-2-7b [12] 68.9% · TriviaQA · EM · % · Llama-2-7b [12] 65.8% · TriviaQA · EM · % · Llama-2-7b [12]	Published observations span 7.9 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #26 of 40 in this displayed comparison; 16.98 pp below the leader.
Mistral 7B v0.1 →Mistral	68.85% 2 reports · range 62.5–75.2%	TriviaQA[9][7] View 2 reports 75.2% · TriviaQA · EM · % · Mistral-7B-v0.1 [9] 62.5% · TriviaQA · EM · % · Mistral-7B-v0.1 [7]	Published observations span 12.7 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #27 of 40 in this displayed comparison; 18.65 pp below the leader.
LLaMA-13B →Meta	67.96% 8 reports · range 56.6–77.9%	TriviaQA[12][13] View 8 reports 77.9% · TriviaQA · EM · % · LLaMA-13B [12] 77.1% · TriviaQA · EM · % · LLaMA-13B [12] 74.4% · TriviaQA · EM · % · LLaMA-13B [12] 70.1% · TriviaQA · EM · % · LLaMA-13B [12] 64% · TriviaQA · EM · % · LLaMA-13B [13] 63.1% · TriviaQA · EM · % · LLaMA-13B [13] 60.5% · TriviaQA · EM · % · LLaMA-13B [13] 56.6% · TriviaQA · EM · % · LLaMA-13B [13]	Published observations span 21.3 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #28 of 40 in this displayed comparison; 19.54 pp below the leader.
Gemma 7B →Google	67.85% 2 reports · range 63.4–72.3%	TriviaQA[9][7] View 2 reports 72.3% · TriviaQA · EM · % · gemma-7b [9] 63.4% · TriviaQA · EM · % · gemma-7b [7]	Published observations span 8.9 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #29 of 40 in this displayed comparison; 19.65 pp below the leader.
InstructGPT 175B →OpenAI	67.83% 9 reports · range 64.3–71.2%	TriviaQA[6][10][11] View 9 reports 71.2% · TriviaQA · EM · % · text-davinci-001 [6] 71.2% · TriviaQA · EM · % · text-davinci-001 [10] 71.2% · TriviaQA · EM · % · text-davinci-001 [11] 68% · TriviaQA · EM · % · text-davinci-001 [6] 68% · TriviaQA · EM · % · text-davinci-001 [10] 68% · TriviaQA · EM · % · text-davinci-001 [11] 64.3% · TriviaQA · EM · % · text-davinci-001 [6] 64.3% · TriviaQA · EM · % · text-davinci-001 [10] 64.3% · TriviaQA · EM · % · text-davinci-001 [11]	Published observations span 6.9 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #30 of 40 in this displayed comparison; 19.67 pp below the leader.
Llama 3-8B →Meta	67.7% 1 report	TriviaQA[9]Meta-Llama-3-8B-InstructReleased 2024-04-18	#31 of 40 in this displayed comparison; 19.8 pp below the leader.
phi-3-mini 3.8B →Microsoft	64% 1 report	TriviaQA[9]Phi-3-mini-4k-instructReleased 2024-04-23	#32 of 40 in this displayed comparison; 23.5 pp below the leader.
Chinchilla →DeepMind	61.37% 3 reports · range 55.4–64.6%	TriviaQA[13] View 3 reports 64.6% · TriviaQA · EM · % · Chinchilla (70B) [13] 64.1% · TriviaQA · EM · % · Chinchilla (70B) [13] 55.4% · TriviaQA · EM · % · Chinchilla (70B) [13]	Published observations span 9.2 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #33 of 40 in this displayed comparison; 26.13 pp below the leader.
LLaMA-7B →Meta	61.18% 8 reports · range 50–71%	TriviaQA[12][13] View 8 reports 71% · TriviaQA · EM · % · LLaMA-7B [12] 70.4% · TriviaQA · EM · % · LLaMA-7B [12] 67.4% · TriviaQA · EM · % · LLaMA-7B [12] 63.3% · TriviaQA · EM · % · LLaMA-7B [12] 57.6% · TriviaQA · EM · % · LLaMA-7B [13] 56.3% · TriviaQA · EM · % · LLaMA-7B [13] 53.4% · TriviaQA · EM · % · LLaMA-7B [13] 50% · TriviaQA · EM · % · LLaMA-7B [13]	Published observations span 21 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #34 of 40 in this displayed comparison; 26.32 pp below the leader.
MPT-7B →MosaicML	59.53% 4 reports · range 55.7–61.6%	TriviaQA[12] View 4 reports 61.6% · TriviaQA · EM · % · mpt-7b [12] 61.2% · TriviaQA · EM · % · mpt-7b [12] 59.6% · TriviaQA · EM · % · mpt-7b [12] 55.7% · TriviaQA · EM · % · mpt-7b [12]	Published observations span 5.9 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #35 of 40 in this displayed comparison; 27.97 pp below the leader.
Falcon-7B →Technology Innovation Institute	58.77% 4 reports · range 52.6–64.6%	TriviaQA[12] View 4 reports 64.6% · TriviaQA · EM · % · falcon-7b [12] 61.1% · TriviaQA · EM · % · falcon-7b [12] 56.8% · TriviaQA · EM · % · falcon-7b [12] 52.6% · TriviaQA · EM · % · falcon-7b [12]	Published observations span 12 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #36 of 40 in this displayed comparison; 28.73 pp below the leader.
phi-3-small 7.4B →Microsoft	58.1% 1 report	TriviaQA[9]Phi-3-small-8k-instructReleased 2024-04-23	#37 of 40 in this displayed comparison; 29.4 pp below the leader.
Gopher (280B) →DeepMind	54.38% 5 reports · range 43.5–57.2%	TriviaQA[13][10][14] View 5 reports 57.2% · TriviaQA · EM · % · Gopher (280B) [13] 57.1% · TriviaQA · EM · % · Gopher (280B) [10] 57.1% · TriviaQA · EM · % · Gopher (280B) [14] 57% · TriviaQA · EM · % · Gopher (280B) [13] 43.5% · TriviaQA · EM · % · Gopher (280B) [13]	Published observations span 13.7 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #38 of 40 in this displayed comparison; 33.12 pp below the leader.
Gemma 2B →Google	53.2% 1 report	TriviaQA[7]gemma-2bReleased 2024-02-21	#39 of 40 in this displayed comparison; 34.3 pp below the leader.
Phi-2 →Microsoft	45.2% 1 report	TriviaQA[9]phi-2Released 2023-12-12	#40 of 40 in this displayed comparison; 42.3 pp below the leader.
BiDAF →TriviaQA · original evaluation · web	40.74% 1 report	TriviaQA · TriviaQA distant supervision · Test exact match[16]TriviaQA · original evaluation · web	Leads this displayed comparison. Source notes Source cell: 40.74
BiDAF →TriviaQA · original evaluation	40.32% 1 report	TriviaQA · TriviaQA distant supervision · Test exact match[16]TriviaQA · original evaluation	#2 of 4 in this displayed comparison; 0.42 pp below the leader. Source notes Source cell: 40.32
Classifier →TriviaQA · original evaluation · web	24% 1 report	TriviaQA · TriviaQA distant supervision · Test exact match[16]TriviaQA · original evaluation · web	#3 of 4 in this displayed comparison; 16.74 pp below the leader. Source notes Source cell: 24.00
Classifier →TriviaQA · original evaluation · Wiki	22.45% 1 report	TriviaQA · TriviaQA distant supervision · Test exact match[16]TriviaQA · original evaluation · Wiki	#4 of 4 in this displayed comparison; 18.29 pp below the leader. Source notes Source cell: 22.45
BiDAF →TriviaQA · original evaluation · web	47.05% 1 report	TriviaQA · TriviaQA distant supervision · Test F1[16]TriviaQA · original evaluation · web	Leads this displayed comparison. Source notes Source cell: 47.05
BiDAF →TriviaQA · original evaluation	45.91% 1 report	TriviaQA · TriviaQA distant supervision · Test F1[16]TriviaQA · original evaluation	#2 of 4 in this displayed comparison; 1.14 pp below the leader. Source notes Source cell: 45.91
Classifier →TriviaQA · original evaluation · web	28.38% 1 report	TriviaQA · TriviaQA distant supervision · Test F1[16]TriviaQA · original evaluation · web	#3 of 4 in this displayed comparison; 18.67 pp below the leader. Source notes Source cell: 28.38
Classifier →TriviaQA · original evaluation · Wiki	26.52% 1 report	TriviaQA · TriviaQA distant supervision · Test F1[16]TriviaQA · original evaluation · Wiki	#4 of 4 in this displayed comparison; 20.53 pp below the leader. Source notes Source cell: 26.52

Sources

Model Card and Evaluations for Claude Models ↗
Anthropic · 2023
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Capabilities & benchmarking — retained external evaluation records ↗
Epoch AI · 2026
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms. Original benchmark URL redirects to an unrelated trending-papers page. The imported observations remain available in Epoch AI’s external-run archive; preserve the original URL in source notes. Originally collected from https://paperswithcode.com/sota/question-answering-on-triviaqa
PaLM 2 Technical Report ↗
Anil, Rohan, Dai, Andrew M., Firat, Orhan, Johnson, Melvin, Lepikhin, Dmitry, Passos, Alexandre et al. · 2023
All 128 authors
Anil, Rohan, Dai, Andrew M., Firat, Orhan, Johnson, Melvin, Lepikhin, Dmitry, Passos, Alexandre, Shakeri, Siamak, Taropa, Emanuel, Bailey, Paige, Chen, Zhifeng, Chu, Eric, Clark, Jonathan H., Shafey, Laurent El, Huang, Yanping, Meier-Hellstern, Kathy, Mishra, Gaurav, Moreira, Erica, Omernick, Mark, Robinson, Kevin, Ruder, Sebastian, Tay, Yi, Xiao, Kefan, Xu, Yuanzhong, Zhang, Yujing, Abrego, Gustavo Hernandez, Ahn, Junwhan, Austin, Jacob, Barham, Paul, Botha, Jan, Bradbury, James, Brahma, Siddhartha, Brooks, Kevin, Catasta, Michele, Cheng, Yong, Cherry, Colin, Choquette-Choo, Christopher A., Chowdhery, Aakanksha, Crepy, Clément, Dave, Shachi, Dehghani, Mostafa, Dev, Sunipa, Devlin, Jacob, Díaz, Mark, Du, Nan, Dyer, Ethan, Feinberg, Vlad, Feng, Fangxiaoyu, Fienber, Vlad, Freitag, Markus, Garcia, Xavier, Gehrmann, Sebastian, Gonzalez, Lucas, Gur-Ari, Guy, Hand, Steven, Hashemi, Hadi, Hou, Le, Howland, Joshua, Hu, Andrea, Hui, Jeffrey, Hurwitz, Jeremy, Isard, Michael, Ittycheriah, Abe, Jagielski, Matthew, Jia, Wenhao, Kenealy, Kathleen, Krikun, Maxim, Kudugunta, Sneha, Lan, Chang, Lee, Katherine, Lee, Benjamin, Li, Eric, Li, Music, Li, Wei, Li, YaGuang, Li, Jian, Lim, Hyeontaek, Lin, Hanzhao, Liu, Zhongtao, Liu, Frederick, Maggioni, Marcello, Mahendru, Aroma, Maynez, Joshua, Misra, Vedant, Moussalem, Maysam, Nado, Zachary, Nham, John, Ni, Eric, Nystrom, Andrew, Parrish, Alicia, Pellat, Marie, Polacek, Martin, Polozov, Alex, Pope, Reiner, Qiao, Siyuan, Reif, Emily, Richter, Bryan, Riley, Parker, Ros, Alex Castro, Roy, Aurko, Saeta, Brennan, Samuel, Rajkumar, Shelby, Renee, Slone, Ambrose, Smilkov, Daniel, So, David R., Sohn, Daniel, Tokumine, Simon, Valter, Dasha, Vasudevan, Vijay, Vodrahalli, Kiran, Wang, Xuezhi, Wang, Pidong, Wang, Zirui, Wang, Tao, Wieting, John, Wu, Yuhuai, Xu, Kelvin, Xu, Yunhan, Xue, Linting, Yin, Pengcheng, Yu, Jiahui, Zhang, Qiao, Zheng, Steven, Zheng, Ce, Zhou, Weikang, Zhou, Denny, Petrov, Slav, Wu, Yonghui
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Llama 2: Open Foundation and Fine-Tuned Chat Models ↗
Touvron, Hugo, Martin, Louis, Stone, Kevin, Albert, Peter, Almahairi, Amjad, Babaei, Yasmine et al. · 2023
All 68 authors
Touvron, Hugo, Martin, Louis, Stone, Kevin, Albert, Peter, Almahairi, Amjad, Babaei, Yasmine, Bashlykov, Nikolay, Batra, Soumya, Bhargava, Prajjwal, Bhosale, Shruti, Bikel, Dan, Blecher, Lukas, Ferrer, Cristian Canton, Chen, Moya, Cucurull, Guillem, Esiobu, David, Fernandes, Jude, Fu, Jeremy, Fu, Wenyin, Fuller, Brian, Gao, Cynthia, Goswami, Vedanuj, Goyal, Naman, Hartshorn, Anthony, Hosseini, Saghar, Hou, Rui, Inan, Hakan, Kardas, Marcin, Kerkez, Viktor, Khabsa, Madian, Kloumann, Isabel, Korenev, Artem, Koura, Punit Singh, Lachaux, Marie-Anne, Lavril, Thibaut, Lee, Jenya, Liskovich, Diana, Lu, Yinghai, Mao, Yuning, Martinet, Xavier, Mihaylov, Todor, Mishra, Pushkar, Molybog, Igor, Nie, Yixin, Poulton, Andrew, Reizenstein, Jeremy, Rungta, Rashi, Saladi, Kalyan, Schelten, Alan, Silva, Ruan, Smith, Eric Michael, Subramanian, Ranjan, Tan, Xiaoqing Ellen, Tang, Binh, Taylor, Ross, Williams, Adina, Kuan, Jian Xiang, Xu, Puxin, Yan, Zheng, Zarov, Iliyan, Zhang, Yuchen, Fan, Angela, Kambadur, Melanie, Narang, Sharan, Rodriguez, Aurelien, Stojnic, Robert, Edunov, Sergey, Scialom, Thomas
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
PaLM: Scaling Language Modeling with Pathways ↗
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts et al. · 2023
All 67 authors
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Language Models are Few-Shot Learners ↗
Brown, Tom B., Mann, Benjamin, Ryder, Nick, Subbiah, Melanie, Kaplan, Jared, Dhariwal, Prafulla et al. · 2020
All 31 authors
Brown, Tom B., Mann, Benjamin, Ryder, Nick, Subbiah, Melanie, Kaplan, Jared, Dhariwal, Prafulla, Neelakantan, Arvind, Shyam, Pranav, Sastry, Girish, Askell, Amanda, Agarwal, Sandhini, Herbert-Voss, Ariel, Krueger, Gretchen, Henighan, Tom, Child, Rewon, Ramesh, Aditya, Ziegler, Daniel M., Wu, Jeffrey, Winter, Clemens, Hesse, Christopher, Chen, Mark, Sigler, Eric, Litwin, Mateusz, Gray, Scott, Chess, Benjamin, Clark, Jack, Berner, Christopher, McCandlish, Sam, Radford, Alec, Sutskever, Ilya, Amodei, Dario
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Gemma: Open Models Based on Gemini Research and Technology ↗
Gemma Team, Mesnard, Thomas, Hardin, Cassidy, Dadashi, Robert, Bhupatiraju, Surya, Pathak, Shreya et al. · 2024
All 108 authors
Gemma Team, Mesnard, Thomas, Hardin, Cassidy, Dadashi, Robert, Bhupatiraju, Surya, Pathak, Shreya, Sifre, Laurent, Rivière, Morgane, Kale, Mihir Sanjay, Love, Juliette, Tafti, Pouya, Hussenot, Léonard, Sessa, Pier Giuseppe, Chowdhery, Aakanksha, Roberts, Adam, Barua, Aditya, Botev, Alex, Castro-Ros, Alex, Slone, Ambrose, Héliou, Amélie, Tacchetti, Andrea, Bulanova, Anna, Paterson, Antonia, Tsai, Beth, Shahriari, Bobak, Lan, Charline Le, Choquette-Choo, Christopher A., Crepy, Clément, Cer, Daniel, Ippolito, Daphne, Reid, David, Buchatskaya, Elena, Ni, Eric, Noland, Eric, Yan, Geng, Tucker, George, Muraru, George-Christian, Rozhdestvenskiy, Grigory, Michalewski, Henryk, Tenney, Ian, Grishchenko, Ivan, Austin, Jacob, Keeling, James, Labanowski, Jane, Lespiau, Jean-Baptiste, Stanway, Jeff, Brennan, Jenny, Chen, Jeremy, Ferret, Johan, Chiu, Justin, Mao-Jones, Justin, Lee, Katherine, Yu, Kathy, Millican, Katie, Sjoesund, Lars Lowe, Lee, Lisa, Dixon, Lucas, Reid, Machel, Mikuła, Maciej, Wirth, Mateo, Sharman, Michael, Chinaev, Nikolai, Thain, Nithum, Bachem, Olivier, Chang, Oscar, Wahltinez, Oscar, Bailey, Paige, Michel, Paul, Yotov, Petko, Chaabouni, Rahma, Comanescu, Ramona, Jana, Reena, Anil, Rohan, McIlroy, Ross, Liu, Ruibo, Mullins, Ryan, Smith, Samuel L, Borgeaud, Sebastian, Girgin, Sertan, Douglas, Sholto, Pandya, Shree, Shakeri, Siamak, De, Soham, Klimenko, Ted, Hennigan, Tom, Feinberg, Vlad, Stokowiec, Wojciech, Chen, Yu-hui, Ahmed, Zafarali, Gong, Zhitao, Warkentin, Tris, Peran, Ludovic, Giang, Minh, Farabet, Clément, Vinyals, Oriol, Dean, Jeff, Kavukcuoglu, Koray, Hassabis, Demis, Ghahramani, Zoubin, Eck, Douglas, Barral, Joelle, Pereira, Fernando, Collins, Eli, Joulin, Armand, Fiedel, Noah, Senter, Evan, Andreev, Alek, Kenealy, Kathleen
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
DeepSeek-V3 Technical Report ↗
DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu et al. · 2024
All 200 authors
DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jiawei Wang, Jin Chen, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Junxiao Song, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Litong Wang, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qiancheng Wang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, Runxin Xu, Ruoyu Zhang, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Shuting Pan, T. Wang, Tao Yun, Tian Pei, Tianyu Sun, W. L. Xiao, Wangding Zeng, Wanjia Zhao, Wei An, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzu Wang, Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojin Shen, Xiaokang Chen, Xiaokang Zhang, Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xingkai Yu, Xinnan Song, Xinxia Shan, Xinyi Zhou, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, Y. K. Li, Y. Q. Wang, Y. X. Wei, Y. X. Zhu, Yang Zhang, Yanhong Xu, Yanhong Xu, Yanping Huang, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang, Yi Yu, Yi Zheng, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Ying Tang, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yu Wu, Yuan Ou, Yuchen Zhu, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yukun Zha, Yunfan Xiong, Yunxian Ma, Yuting Yan, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Z. F. Wu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhen Huang, Zhen Zhang, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhibin Gou, Zhicheng Ma, Zhigang Yan, Zhihong Shao, Zhipeng Xu, Zhiyu Wu, Zhongyu Zhang, Zhuoshu Li, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Ziyi Gao, Zizheng Pan
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone ↗
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach et al. · 2024
All 129 authors
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song, Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang, Guanhua Wang, Lijuan Wang, Shuohang Wang, Xin Wang, Yu Wang, Rachel Ward, Wen Wen, Philipp Witte, Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Jilong Xue, Sonali Yadav, Fan Yang, Jianwei Yang, Yifan Yang, Ziyi Yang, Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, Xiren Zhou
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts ↗
Du, Nan, Huang, Yanping, Dai, Andrew M., Tong, Simon, Lepikhin, Dmitry, Xu, Yuanzhong et al. · 2021
All 27 authors
Du, Nan, Huang, Yanping, Dai, Andrew M., Tong, Simon, Lepikhin, Dmitry, Xu, Yuanzhong, Krikun, Maxim, Zhou, Yanqi, Yu, Adams Wei, Firat, Orhan, Zoph, Barret, Fedus, Liam, Bosma, Maarten, Zhou, Zongwei, Wang, Tao, Wang, Yu Emma, Webster, Kellie, Pellat, Marie, Robinson, Kevin, Meier-Hellstern, Kathleen, Duke, Toju, Dixon, Lucas, Zhang, Kun, Le, Quoc V, Wu, Yonghui, Chen, Zhifeng, Cui, Claire
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Language Models are Few-Shot Learners ↗
Brown, Tom B., Mann, Benjamin, Ryder, Nick, Subbiah, Melanie, Kaplan, Jared, Dhariwal, Prafulla et al. · 2020
All 31 authors
Brown, Tom B., Mann, Benjamin, Ryder, Nick, Subbiah, Melanie, Kaplan, Jared, Dhariwal, Prafulla, Neelakantan, Arvind, Shyam, Pranav, Sastry, Girish, Askell, Amanda, Agarwal, Sandhini, Herbert-Voss, Ariel, Krueger, Gretchen, Henighan, Tom, Child, Rewon, Ramesh, Aditya, Ziegler, Daniel M., Wu, Jeffrey, Winter, Clemens, Hesse, Christopher, Chen, Mark, Sigler, Eric, Litwin, Mateusz, Gray, Scott, Chess, Benjamin, Clark, Jack, Berner, Christopher, McCandlish, Sam, Radford, Alec, Sutskever, Ilya, Amodei, Dario
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Llama 2: Open Foundation and Fine-Tuned Chat Models ↗
Touvron, Hugo, Martin, Louis, Stone, Kevin, Albert, Peter, Almahairi, Amjad, Babaei, Yasmine et al. · 2023
All 68 authors
Touvron, Hugo, Martin, Louis, Stone, Kevin, Albert, Peter, Almahairi, Amjad, Babaei, Yasmine, Bashlykov, Nikolay, Batra, Soumya, Bhargava, Prajjwal, Bhosale, Shruti, Bikel, Dan, Blecher, Lukas, Ferrer, Cristian Canton, Chen, Moya, Cucurull, Guillem, Esiobu, David, Fernandes, Jude, Fu, Jeremy, Fu, Wenyin, Fuller, Brian, Gao, Cynthia, Goswami, Vedanuj, Goyal, Naman, Hartshorn, Anthony, Hosseini, Saghar, Hou, Rui, Inan, Hakan, Kardas, Marcin, Kerkez, Viktor, Khabsa, Madian, Kloumann, Isabel, Korenev, Artem, Koura, Punit Singh, Lachaux, Marie-Anne, Lavril, Thibaut, Lee, Jenya, Liskovich, Diana, Lu, Yinghai, Mao, Yuning, Martinet, Xavier, Mihaylov, Todor, Mishra, Pushkar, Molybog, Igor, Nie, Yixin, Poulton, Andrew, Reizenstein, Jeremy, Rungta, Rashi, Saladi, Kalyan, Schelten, Alan, Silva, Ruan, Smith, Eric Michael, Subramanian, Ranjan, Tan, Xiaoqing Ellen, Tang, Binh, Taylor, Ross, Williams, Adina, Kuan, Jian Xiang, Xu, Puxin, Yan, Zheng, Zarov, Iliyan, Zhang, Yuchen, Fan, Angela, Kambadur, Melanie, Narang, Sharan, Rodriguez, Aurelien, Stojnic, Robert, Edunov, Sergey, Scialom, Thomas
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
LLaMA: Open and Efficient Foundation Language Models ↗
Touvron, Hugo, Lavril, Thibaut, Izacard, Gautier, Martinet, Xavier, Lachaux, Marie-Anne, Lacroix, Timothée et al. · 2023
All 14 authors
Touvron, Hugo, Lavril, Thibaut, Izacard, Gautier, Martinet, Xavier, Lachaux, Marie-Anne, Lacroix, Timothée, Rozière, Baptiste, Goyal, Naman, Hambro, Eric, Azhar, Faisal, Rodriguez, Aurelien, Joulin, Armand, Grave, Edouard, Lample, Guillaume
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Scaling Language Models: Methods, Analysis & Insights from Training Gopher ↗
Rae, Jack W., Borgeaud, Sebastian, Cai, Trevor, Millican, Katie, Hoffmann, Jordan, Song, Francis et al. · 2021
All 80 authors
Rae, Jack W., Borgeaud, Sebastian, Cai, Trevor, Millican, Katie, Hoffmann, Jordan, Song, Francis, Aslanides, John, Henderson, Sarah, Ring, Roman, Young, Susannah, Rutherford, Eliza, Hennigan, Tom, Menick, Jacob, Cassirer, Albin, Powell, Richard, Driessche, George van den, Hendricks, Lisa Anne, Rauh, Maribeth, Huang, Po-Sen, Glaese, Amelia, Welbl, Johannes, Dathathri, Sumanth, Huang, Saffron, Uesato, Jonathan, Mellor, John, Higgins, Irina, Creswell, Antonia, McAleese, Nat, Wu, Amy, Elsen, Erich, Jayakumar, Siddhant, Buchatskaya, Elena, Budden, David, Sutherland, Esme, Simonyan, Karen, Paganini, Michela, Sifre, Laurent, Martens, Lena, Li, Xiang Lorraine, Kuncoro, Adhiguna, Nematzadeh, Aida, Gribovskaya, Elena, Donato, Domenic, Lazaridou, Angeliki, Mensch, Arthur, Lespiau, Jean-Baptiste, Tsimpoukelli, Maria, Grigorev, Nikolai, Fritz, Doug, Sottiaux, Thibault, Pajarskas, Mantas, Pohlen, Toby, Gong, Zhitao, Toyama, Daniel, d'Autume, Cyprien de Masson, Li, Yujia, Terzi, Tayfun, Mikulik, Vladimir, Babuschkin, Igor, Clark, Aidan, Casas, Diego de Las, Guy, Aurelia, Jones, Chris, Bradbury, James, Johnson, Matthew, Hechtman, Blake, Weidinger, Laura, Gabriel, Iason, Isaac, William, Lockhart, Ed, Osindero, Simon, Rimell, Laura, Dyer, Chris, Vinyals, Oriol, Ayoub, Kareem, Stanway, Jeff, Bennett, Lorrayne, Hassabis, Demis, Kavukcuoglu, Koray, Irving, Geoffrey
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
Releasing Claude Instant 1.2 ↗
Anthropic
Dates & evaluation details
TriviaQA · EM · % · 2026-10-26
Epoch AI public data snapshot, CC BY 4.0. Externally collected result; original evaluation provider is linked. CSV: trivia_qa_external.csv; score column: EM; fraction scale: 1. Model version is retained in configuration. Original source authors retain credit and any additional licensing terms.
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension — published result table ↗
Joshi, Mandar, Choi, Eunsol, Weld, Daniel S., Zettlemoyer, Luke · 2017
Dates & evaluation details
TriviaQA · TriviaQA distant supervision · Test exact match · 2026-10-27
Published model comparison, 2017. TriviaQA · original evaluation. Native TriviaQA test result under distant supervision. Web/Wikipedia domains remain source settings; verified-test and Natural Questions are separate datasets. Table 7; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.
TriviaQA · TriviaQA distant supervision · Test F1 · 2026-10-27
Published model comparison, 2017. TriviaQA · original evaluation. Native TriviaQA test result under distant supervision. Web/Wikipedia domains remain source settings; verified-test and Natural Questions are separate datasets. Table 7; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

TRACE THE SCORE · Per-model notes use published breakdowns where available.

Score vs. release date

Epoch AI · 115 observations

TriviaQA · EM · % · higher is better

Dates are model release dates recorded by Epoch AI, not evaluation dates. Each point retains its published score and configuration. Separate releases are not a matched longitudinal experiment. Missing dates are omitted. Explore all reported results ↓

The dashed line traces the best observed score through the model release dates. It describes this selection of reports, with their original configurations. It does not estimate unmeasured models or costs.

View plotted values and sources (115)

TriviaQA · TriviaQA · EM · %. Published measurements in their original score units.
Model	Score	Model release date	Configuration & source
Claude 2 →	87.5%	2023-07-11	claude-2.0 Epoch AI · TriviaQA · EM · % ↗
Claude 1.3 →	86.7%	2023-04-18	claude-1.3 Epoch AI · TriviaQA · EM · % ↗
Claude Instant →	78.9%	2023-08-09	claude-instant-1.1 Epoch AI · TriviaQA · EM · % ↗
GPT-4 (Jun 2023) →	84.8%	2023-06-13	gpt-4-0613 Epoch AI · TriviaQA · EM · % ↗
PaLM 2-L →	86.1%	2023-05-17	PaLM 2-L Epoch AI · TriviaQA · EM · % ↗
PaLM 2-M →	81.7%	2023-05-17	PaLM 2-M Epoch AI · TriviaQA · EM · % ↗
PaLM 2-S →	75.2%	2023-05-17	PaLM 2-S Epoch AI · TriviaQA · EM · % ↗
Llama 2-70B →	85%	2023-07-18	Llama-2-70b-hf Epoch AI · TriviaQA · EM · % ↗
PaLM (540B) →	81.4%	2022-04-04	PaLM 540B Epoch AI · TriviaQA · EM · % ↗
PaLM (540B) →	81.4%	2022-04-04	PaLM 540B Epoch AI · TriviaQA · EM · % ↗
PaLM (540B) →	76.9%	2022-04-04	PaLM 540B Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	71.2%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	68%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	64.3%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
Llama 2-13B →	79.6%	2023-07-18	Llama-2-13b Epoch AI · TriviaQA · EM · % ↗
Llama 2-7B →	72.1%	2023-07-18	Llama-2-7b Epoch AI · TriviaQA · EM · % ↗
Gemma 7B →	63.4%	2024-02-21	gemma-7b Epoch AI · TriviaQA · EM · % ↗
Mistral 7B v0.1 →	62.5%	2023-09-27	Mistral-7B-v0.1 Epoch AI · TriviaQA · EM · % ↗
Gemma 2B →	53.2%	2024-02-21	gemma-2b Epoch AI · TriviaQA · EM · % ↗
DeepSeek-V3 →	82.9%	2024-12-26	DeepSeek-V3 Epoch AI · TriviaQA · EM · % ↗
Llama 3.1-405B →	82.7%	2024-07-23	Llama-3.1-405B Epoch AI · TriviaQA · EM · % ↗
DeepSeek-V2 (MoE-236B, May 2024) →	80%	2024-05-07	DeepSeek-V2 Epoch AI · TriviaQA · EM · % ↗
Qwen2.5-72B →	71.9%	2024-09-19	Qwen2.5-72B Epoch AI · TriviaQA · EM · % ↗
GPT-3.5 Turbo (Nov 2023) →	85.8%	2023-11-06	gpt-3.5-turbo-1106 Epoch AI · TriviaQA · EM · % ↗
Mixtral 8x7B →	82.2%	2023-12-11	Mixtral-8x7B-v0.1 Epoch AI · TriviaQA · EM · % ↗
Mistral 7B v0.1 →	75.2%	2023-09-27	Mistral-7B-v0.1 Epoch AI · TriviaQA · EM · % ↗
phi-3-medium 14B →	73.9%	2024-04-23	Phi-3-medium-128k-instruct Epoch AI · TriviaQA · EM · % ↗
Gemma 7B →	72.3%	2024-02-21	gemma-7b Epoch AI · TriviaQA · EM · % ↗
Llama 3-8B →	67.7%	2024-04-18	Meta-Llama-3-8B-Instruct Epoch AI · TriviaQA · EM · % ↗
phi-3-mini 3.8B →	64%	2024-04-23	Phi-3-mini-4k-instruct Epoch AI · TriviaQA · EM · % ↗
phi-3-small 7.4B →	58.1%	2024-04-23	Phi-3-small-8k-instruct Epoch AI · TriviaQA · EM · % ↗
Phi-2 →	45.2%	2023-12-12	phi-2 Epoch AI · TriviaQA · EM · % ↗
GLaM →	75.8%	2021-12-13	GLaM (MoE) Epoch AI · TriviaQA · EM · % ↗
GLaM →	75.8%	2021-12-13	GLaM (MoE) Epoch AI · TriviaQA · EM · % ↗
GLaM →	71.3%	2021-12-13	GLaM (MoE) Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	71.2%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	68%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	64.3%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
Gopher (280B) →	57.1%	2021-12-08	Gopher (280B) Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	71.2%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	68%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
InstructGPT 175B →	64.3%	2022-01-27	text-davinci-001 Epoch AI · TriviaQA · EM · % ↗
Llama 2-70B →	87.6%	2023-07-18	Llama-2-70b-hf Epoch AI · TriviaQA · EM · % ↗
Llama 2-70B →	87.5%	2023-07-18	Llama-2-70b-hf Epoch AI · TriviaQA · EM · % ↗
LLaMA-65B →	86%	2023-02-24	LLaMA-65B Epoch AI · TriviaQA · EM · % ↗
LLaMA-65B →	85.9%	2023-02-24	LLaMA-65B Epoch AI · TriviaQA · EM · % ↗
Llama 2-70B →	85%	2023-07-18	Llama-2-70b-hf Epoch AI · TriviaQA · EM · % ↗
Llama 2-34B →	84.6%	2023-07-18	Llama-2-34b Epoch AI · TriviaQA · EM · % ↗
LLaMA-65B →	84.5%	2023-02-24	LLaMA-65B Epoch AI · TriviaQA · EM · % ↗
Llama 2-34B →	84.5%	2023-07-18	Llama-2-34b Epoch AI · TriviaQA · EM · % ↗
LLaMA-33B →	83.8%	2023-02-24	LLaMA-33B Epoch AI · TriviaQA · EM · % ↗
LLaMA-33B →	83.6%	2023-02-24	LLaMA-33B Epoch AI · TriviaQA · EM · % ↗
Llama 2-34B →	83.3%	2023-07-18	Llama-2-34b Epoch AI · TriviaQA · EM · % ↗
Llama 2-70B →	82.4%	2023-07-18	Llama-2-70b-hf Epoch AI · TriviaQA · EM · % ↗
LLaMA-65B →	81.7%	2023-02-24	LLaMA-65B Epoch AI · TriviaQA · EM · % ↗
Llama 2-34B →	81%	2023-07-18	Llama-2-34b Epoch AI · TriviaQA · EM · % ↗
LLaMA-33B →	80.7%	2023-02-24	LLaMA-33B Epoch AI · TriviaQA · EM · % ↗
Falcon-40B →	79.9%	2023-03-15	falcon-40b Epoch AI · TriviaQA · EM · % ↗
Llama 2-13B →	79.6%	2023-07-18	Llama-2-13b Epoch AI · TriviaQA · EM · % ↗
Falcon-40B →	79.6%	2023-03-15	falcon-40b Epoch AI · TriviaQA · EM · % ↗
Llama 2-13B →	79.4%	2023-07-18	Llama-2-13b Epoch AI · TriviaQA · EM · % ↗
LLaMA-33B →	78.7%	2023-02-24	LLaMA-33B Epoch AI · TriviaQA · EM · % ↗
Falcon-40B →	78.6%	2023-03-15	falcon-40b Epoch AI · TriviaQA · EM · % ↗
LLaMA-13B →	77.9%	2023-02-24	LLaMA-13B Epoch AI · TriviaQA · EM · % ↗
Llama 2-13B →	77.2%	2023-07-18	Llama-2-13b Epoch AI · TriviaQA · EM · % ↗
LLaMA-13B →	77.1%	2023-02-24	LLaMA-13B Epoch AI · TriviaQA · EM · % ↗
Falcon-40B →	74.6%	2023-03-15	falcon-40b Epoch AI · TriviaQA · EM · % ↗
LLaMA-13B →	74.4%	2023-02-24	LLaMA-13B Epoch AI · TriviaQA · EM · % ↗
Llama 2-7B →	73.7%	2023-07-18	Llama-2-7b Epoch AI · TriviaQA · EM · % ↗
MPT-30B →	73.6%	2023-06-22	mpt-30b Epoch AI · TriviaQA · EM · % ↗
MPT-30B →	73.3%	2023-06-22	mpt-30b Epoch AI · TriviaQA · EM · % ↗
Llama 2-13B →	73.1%	2023-07-18	Llama-2-13b Epoch AI · TriviaQA · EM · % ↗
Llama 2-7B →	72.1%	2023-07-18	Llama-2-7b Epoch AI · TriviaQA · EM · % ↗
MPT-30B →	71.3%	2023-06-22	mpt-30b Epoch AI · TriviaQA · EM · % ↗
LLaMA-7B →	71%	2023-02-24	LLaMA-7B Epoch AI · TriviaQA · EM · % ↗
LLaMA-7B →	70.4%	2023-02-24	LLaMA-7B Epoch AI · TriviaQA · EM · % ↗
LLaMA-13B →	70.1%	2023-02-24	LLaMA-13B Epoch AI · TriviaQA · EM · % ↗
Llama 2-7B →	68.9%	2023-07-18	Llama-2-7b Epoch AI · TriviaQA · EM · % ↗
MPT-30B →	68%	2023-06-22	mpt-30b Epoch AI · TriviaQA · EM · % ↗
LLaMA-7B →	67.4%	2023-02-24	LLaMA-7B Epoch AI · TriviaQA · EM · % ↗
Llama 2-7B →	65.8%	2023-07-18	Llama-2-7b Epoch AI · TriviaQA · EM · % ↗
Falcon-7B →	64.6%	2023-04-24	falcon-7b Epoch AI · TriviaQA · EM · % ↗
LLaMA-7B →	63.3%	2023-02-24	LLaMA-7B Epoch AI · TriviaQA · EM · % ↗
MPT-7B →	61.6%	2023-05-05	mpt-7b Epoch AI · TriviaQA · EM · % ↗
MPT-7B →	61.2%	2023-05-05	mpt-7b Epoch AI · TriviaQA · EM · % ↗
Falcon-7B →	61.1%	2023-04-24	falcon-7b Epoch AI · TriviaQA · EM · % ↗
MPT-7B →	59.6%	2023-05-05	mpt-7b Epoch AI · TriviaQA · EM · % ↗
Falcon-7B →	56.8%	2023-04-24	falcon-7b Epoch AI · TriviaQA · EM · % ↗
MPT-7B →	55.7%	2023-05-05	mpt-7b Epoch AI · TriviaQA · EM · % ↗
Falcon-7B →	52.6%	2023-04-24	falcon-7b Epoch AI · TriviaQA · EM · % ↗
LLaMA-65B →	73%	2023-02-24	LLaMA-65B Epoch AI · TriviaQA · EM · % ↗
LLaMA-65B →	72.6%	2023-02-24	LLaMA-65B Epoch AI · TriviaQA · EM · % ↗
LLaMA-65B →	71.6%	2023-02-24	LLaMA-65B Epoch AI · TriviaQA · EM · % ↗
LLaMA-33B →	70.4%	2023-02-24	LLaMA-33B Epoch AI · TriviaQA · EM · % ↗
LLaMA-33B →	69.9%	2023-02-24	LLaMA-33B Epoch AI · TriviaQA · EM · % ↗
LLaMA-65B →	68.2%	2023-02-24	LLaMA-65B Epoch AI · TriviaQA · EM · % ↗
LLaMA-33B →	67.9%	2023-02-24	LLaMA-33B Epoch AI · TriviaQA · EM · % ↗
LLaMA-33B →	65.1%	2023-02-24	LLaMA-33B Epoch AI · TriviaQA · EM · % ↗
Chinchilla →	64.6%	2022-03-29	Chinchilla (70B) Epoch AI · TriviaQA · EM · % ↗
Chinchilla →	64.1%	2022-03-29	Chinchilla (70B) Epoch AI · TriviaQA · EM · % ↗
LLaMA-13B →	64%	2023-02-24	LLaMA-13B Epoch AI · TriviaQA · EM · % ↗
LLaMA-13B →	63.1%	2023-02-24	LLaMA-13B Epoch AI · TriviaQA · EM · % ↗
LLaMA-13B →	60.5%	2023-02-24	LLaMA-13B Epoch AI · TriviaQA · EM · % ↗
LLaMA-7B →	57.6%	2023-02-24	LLaMA-7B Epoch AI · TriviaQA · EM · % ↗
Gopher (280B) →	57.2%	2021-12-08	Gopher (280B) Epoch AI · TriviaQA · EM · % ↗
Gopher (280B) →	57%	2021-12-08	Gopher (280B) Epoch AI · TriviaQA · EM · % ↗
LLaMA-13B →	56.6%	2023-02-24	LLaMA-13B Epoch AI · TriviaQA · EM · % ↗
LLaMA-7B →	56.3%	2023-02-24	LLaMA-7B Epoch AI · TriviaQA · EM · % ↗
Chinchilla →	55.4%	2022-03-29	Chinchilla (70B) Epoch AI · TriviaQA · EM · % ↗
LLaMA-7B →	53.4%	2023-02-24	LLaMA-7B Epoch AI · TriviaQA · EM · % ↗
LLaMA-7B →	50%	2023-02-24	LLaMA-7B Epoch AI · TriviaQA · EM · % ↗
Gopher (280B) →	43.5%	2021-12-08	Gopher (280B) Epoch AI · TriviaQA · EM · % ↗
Gopher (280B) →	57.1%	2021-12-08	Gopher (280B) Epoch AI · TriviaQA · EM · % ↗
Claude Instant →	78.9%	2023-08-09	claude-instant-1.1 Epoch AI · TriviaQA · EM · % ↗
Claude Instant →	78.7%	2023-08-09	claude-instant-1.2 Epoch AI · TriviaQA · EM · % ↗

Epoch data and evaluation setting

Epoch AI publishes this benchmark in its capability collection. Imported rows retain the published score column and model configuration. Original authors and evaluation providers remain credited in the result references.

How to read the result

Compare the named task version, tools and scoring rule. Epoch-run evaluations and externally collected results have different provenance. ECI inclusion does not make a benchmark score an ECI value.

Epoch attribution and score coverage

Data is attributed to Epoch AI under CC BY 4.0, with original evaluation providers credited in the linked source snapshots. Scores are imported only where a published mapping identifies the field and its scale; missing or incompatible records are omitted rather than assigned zero.

Versions and evaluation settings

The original-paper observations are TriviaQA. Consolidate on TriviaQA, retaining distant-supervision exact match and F1 as different measures.

Where this fits in an evaluation suite

Read accuracy together with error and abstention rates. Domain averages can hide language gaps, specialist failures, and poorly calibrated confidence.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Retain Epoch’s task version, source score column, model version, tool setting and sampling budget. External observations retain their original publication URL.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.

PRIMARY EVIDENCE & FURTHER READING

Epoch AI · benchmark and methodology ↗
Capabilities & benchmarking — retained external evaluation records ↗
Epoch AI · 2026
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension — published result table ↗
Joshi, Mandar, Choi, Eunsol, Weld, Daniel S., Zettlemoyer, Luke · 2017

Reviewed 2026-10-26. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.

Continue exploring

All knowledge & truth benchmarks →