L-Eval & LongBench (v1)

First-generation standardized bilingual long-context evaluation suites (4k–64k tokens).26 Oct 2026

One row per model and compatible score measure. Mean = equal-weight arithmetic average of imported reports. Distinct configurations reported for the same model in a source table remain separate. Reports can overlap; the range is not a confidence interval. Unbundle to inspect each observation.
		Evidence	What the result shows
GPT-4-32k →OpenAI	73.11 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Ret.: ✗; Tokens: 32k	Leads this displayed comparison. Source notes Source cell: 73.11
Turbo-16k-0613 →	60.73 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Ret.: ✗; Tokens: 16k	#2 of 23 in this displayed comparison; 12.38 points below the leader. Source notes Source cell: 60.73
AdaEmb-Turbo-4k-0613 →	49.73 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Ret.: ✓; Tokens: 4k	#3 of 23 in this displayed comparison; 23.38 points below the leader. Source notes Source cell: 49.73
BM25-Turbo-4k-0613 →	49.65 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Ret.: ✓; Tokens: 4k	#4 of 23 in this displayed comparison; 23.46 points below the leader. Source notes Source cell: 49.65
Vicuna1.5-13b-16k →	43.46 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 16k	#5 of 23 in this displayed comparison; 29.65 points below the leader. Source notes Source cell: 43.46↑
Chatglm2-6b-32k →Z.ai	39.01 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 32k	#6 of 23 in this displayed comparison; 34.1 points below the leader. Source notes Source cell: 39.01↑
Llama2-13b-chat →	39.01 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the pretraining context length · Ret.: ✗; Tokens: 4k	#6 of 23 in this displayed comparison; 34.1 points below the leader. Source notes Source cell: 39.01
Vicuna1.5-7b-16k →	36.39 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 16k	#8 of 23 in this displayed comparison; 36.72 points below the leader. Source notes Source cell: 36.39↑
Chatglm2-6b-8k →Z.aiL-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 8k	35.95 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 8k	#9 of 23 in this displayed comparison; 37.16 points below the leader. Source notes Source cell: 35.95
Llama2-13b-NTK(Dyn)* →	35.87 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 16k	#10 of 23 in this displayed comparison; 37.24 points below the leader. Source notes Source cell: 35.87
Longchat-13b-16k →	34.9 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 16k	#11 of 23 in this displayed comparison; 38.21 points below the leader. Source notes Source cell: 34.90
Chatglm2-6b-8k →Z.aiL-Eval · exam evaluation · Truncating input tokens to the pretraining context length · Ret.: ✗; Tokens: 2k	34.69 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the pretraining context length · Ret.: ✗; Tokens: 2k	#12 of 23 in this displayed comparison; 38.42 points below the leader. Source notes Source cell: 34.69
Llama2-7b-NTK* →	33.74 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 16k	#13 of 23 in this displayed comparison; 39.37 points below the leader. Source notes Source cell: 33.74
Llama2-13b-NTK* →	33.69 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 16k	#14 of 23 in this displayed comparison; 39.42 points below the leader. Source notes Source cell: 33.69
Llama2-7b-chat →	33.12 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the pretraining context length · Ret.: ✗; Tokens: 4k	#15 of 23 in this displayed comparison; 39.99 points below the leader. Source notes Source cell: 33.12
Longchat-7b-16k →	31.58 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 16k	#16 of 23 in this displayed comparison; 41.53 points below the leader. Source notes Source cell: 31.58
Longchat1.5-7b-32k →	31.45 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 32k	#17 of 23 in this displayed comparison; 41.66 points below the leader. Source notes Source cell: 31.45
Vicuna1.3-7b-2k →	30.01 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the pretraining context length · Ret.: ✗; Tokens: 2k	#18 of 23 in this displayed comparison; 43.1 points below the leader. Source notes Source cell: 30.01
XGen-7b-8k →	27.63 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 8k	#19 of 23 in this displayed comparison; 45.48 points below the leader. Source notes Source cell: 27.63
XGen-7b-8k (2k-4k-8k) →	26.41 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the pretraining context length · Ret.: ✗; Tokens: 2k	#20 of 23 in this displayed comparison; 46.7 points below the leader. Source notes Source cell: 26.41
Llama2-7b-4k (w/o SFT) →	19.31 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the pretraining context length · Ret.: ✗; Tokens: 4k	#21 of 23 in this displayed comparison; 53.8 points below the leader. Source notes Source cell: 19.31
Llama1-7b-2k (w/o SFT) →	19.22 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the pretraining context length · Ret.: ✗; Tokens: 2k	#22 of 23 in this displayed comparison; 53.89 points below the leader. Source notes Source cell: 19.22
MPT-7b-65k →	19.22 points 1 report	L-Eval & LongBench (v1) · L-Eval · closed-ended average[1]L-Eval · exam evaluation · Truncating input tokens to the further finetuning context length · Ret.: ✗; Tokens: 8k	#22 of 23 in this displayed comparison; 53.89 points below the leader. Source notes Source cell: 19.22

Sources

L-Eval: Instituting Standardized Evaluation for Long Context Language Models — published result table ↗
Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang et al. · 2023
All 8 authors
Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, Xipeng Qiu
Dates & evaluation details
L-Eval & LongBench (v1) · closed-ended average · exam evaluation · 2026-10-26
Published model comparison, 2023. L-Eval · exam evaluation. L-Eval · closed-ended average; L-Eval · exam evaluation. The source’s task split, model configuration and scoring units are retained. Table 3; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

TRACE THE SCORE · Per-model notes use published breakdowns where available.

What it measures

First-generation standardized bilingual long-context evaluation suites (4k–64k tokens).

Reading the measurement

F1 balances precision and recall. Check both: a system can miss relevant evidence or include unsupported material, and those errors have different consequences. Keep the matching rule, averaging policy and denominator fixed.

Published evaluation

L-Eval · closed-ended average; L-Eval · exam evaluation. The source’s task split, model configuration and scoring units are retained.

Where this fits in an evaluation suite

A large context window is a capacity specification. Successful retrieval and cross-document reasoning are observed capabilities. Test both at the actual lengths your workflow needs.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

L-Eval · closed-ended average; L-Eval · exam evaluation. The source’s task split, model configuration and scoring units are retained.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.

PRIMARY EVIDENCE & FURTHER READING

L-Eval: Instituting Standardized Evaluation for Long Context Language Models — published result table ↗
Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang et al. · 2023
All 8 authors
Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, Xipeng Qiu

Reviewed 2026-10-26. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.

Continue exploring

All long-context understanding benchmarks →

Sources

What it measures

Reading the measurement

Published evaluation

Where this fits in an evaluation suite

A reproducible comparison

Continue exploring

GDP.pdf

AA-LCR v1.1

MLCR-AA

RULER