MMLU ↗81%
Gemini 1.5 Pro (0409 preview) · MMLU · EM · % ↗
Google / MODEL EVIDENCE
Gemini 1.5 Pro (0409 preview)
2 benchmarks · 6 observations · 2 sources
Scores at a glance
2 benchmarks · large dot: Gemini 1.5 Pro (0409 preview); small dots: closest models where reported
Gemini 1.5 Pro (0409 preview) · MMLU-Redux (2.0) · Virology · corrected-instance accuracy · MMLU-Redux 2.0 paper · audited subject comparison ↗
4 more reported results
92% · Gemini 1.5 Pro (0409 preview) · MMLU-Redux (2.0) · Logical Fallacies · corrected-instance accuracy · MMLU-Redux 2.0 paper · audited subject comparison · Are We Done with MMLU? ↗
67% · Gemini 1.5 Pro (0409 preview) · MMLU-Redux (2.0) · College Chemistry · corrected-instance accuracy · MMLU-Redux 2.0 paper · audited subject comparison · Are We Done with MMLU? ↗
61% · Gemini 1.5 Pro (0409 preview) · MMLU-Redux (2.0) · Professional Law · corrected-instance accuracy · MMLU-Redux 2.0 paper · audited subject comparison · Are We Done with MMLU? ↗
95% · Gemini 1.5 Pro (0409 preview) · MMLU-Redux (2.0) · Human Sexuality · corrected-instance accuracy · MMLU-Redux 2.0 paper · audited subject comparison · Are We Done with MMLU? ↗
Closest models by score
Similar positions on shared benchmark leaderboards. Smaller gaps are closer.
There are not enough shared scored settings to identify close comparisons reliably.
Sources
Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.
- Are We Done with MMLU? — published result table ↗
All 16 authors
Gema, Aryo Pradipta, Leang, Joshua Ong Jun, Hong, Giwon, Devoto, Alessio, Mancino, Alberto Carlo Maria, Saxena, Rohit, He, Xuanli, Zhao, Yu, Du, Xiaotang, Madani, Mohammad Reza Ghasemi, Barale, Claire, McHardy, Robert, Harris, Joshua, Kaddour, Jean, van Krieken, Emile, Minervini, Pasquale
- Holistic Evaluation of Language Models (HELM) ↗
Names used in source tables
Gemini 1.5 Pro (0409 preview)