What it measures
Foundational 80-prompt two-turn conversational instruction-following benchmark across 8 categories (and 1,388-dialog 10-turn extension).
Reading the measurement
Separate satisfying instructions from factual accuracy, human preference and safe behavior. Inspect strict all-constraint success as well as average criterion credit. Judge choice and adversarial prompt selection can materially change the result.
Published evaluation
Published multi-turn judge score on the source 1–10 scale. MT-Bench-101 is a separate protocol; this result covers the original MT-Bench.
Where this fits in an evaluation suite
Format compliance, preference, truthfulness, and safety require separate measurements. Keep desirable writing style from masking factual errors.
Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.
A reproducible comparison
Published multi-turn judge score on the source 1–10 scale. MT-Bench-101 is a separate protocol; this result covers the original MT-Bench.
Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.
PRIMARY EVIDENCE & FURTHER READING
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — published result table ↗
All 13 authors
Zheng, Lianmin, Chiang, Wei-Lin, Sheng, Ying, Zhuang, Siyuan, Wu, Zhanghao, Zhuang, Yonghao, Lin, Zi, Li, Zhuohan, Li, Dacheng, Xing, Eric P., Zhang, Hao, Gonzalez, Joseph E., Stoica, Ion
Reviewed 2026-10-27. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.