RLTasksby ulam.ai
Models / Reported model

Reported model / MODEL EVIDENCE

PaLM 8B

4 benchmarks · 7 observations · 3 sources

Scores at a glance

4 benchmarks · large dot: PaLM 8B; small dots: closest models where reported

MultiMedQA & HealthSearchQA ↗25.7%
25.760.3

PaLM 8B · MultiMedQA & HealthSearchQA · MedQA four-option · five-shot · accuracy · PaLM and Flan-PaLM comparison ↗

2 more reported results

26.7% · PaLM 8B · MultiMedQA & HealthSearchQA · MedMCQA · five-shot · accuracy · PaLM and Flan-PaLM comparison · Large Language Models Encode Clinical Knowledge ↗

34% · PaLM 8B · MultiMedQA & HealthSearchQA · PubMedQA · three-shot · accuracy · PaLM and Flan-PaLM comparison · Large Language Models Encode Clinical Knowledge ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

44.8 percentile-point gap

3 of 4 shared benchmark comparisons

74.9 percentile-point gap

3 of 4 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. LLaMA: Open and Efficient Foundation Language Models ↗

    Touvron, Hugo, Lavril, Thibaut, Izacard, Gautier, Martinet, Xavier, Lachaux, Marie-Anne, Lacroix, Timothée et al. · 2023

    All 14 authors

    Touvron, Hugo, Lavril, Thibaut, Izacard, Gautier, Martinet, Xavier, Lachaux, Marie-Anne, Lacroix, Timothée, Rozière, Baptiste, Goyal, Naman, Hambro, Eric, Azhar, Faisal, Rodriguez, Aurelien, Joulin, Armand, Grave, Edouard, Lample, Guillaume

  2. Large Language Models Encode Clinical Knowledge — published result table ↗

    Singhal, Karan, Azizi, Shekoofeh, Tu, Tao, Mahdavi, S. Sara, Wei, Jason, Chung, Hyung Won et al. · 2022

    All 30 authors

    Singhal, Karan, Azizi, Shekoofeh, Tu, Tao, Mahdavi, S. Sara, Wei, Jason, Chung, Hyung Won, Scales, Nathan, Tanwani, Ajay, Cole-Lewis, Heather, Pfohl, Stephen, Payne, Perry, Seneviratne, Martin, Gamble, Paul, Kelly, Chris, Scharli, Nathaneal, Chowdhery, Aakanksha, Mansfield, Philip, Arcas, Blaise Aguera y, Webster, Dale, Corrado, Greg S., Matias, Yossi, Chou, Katherine, Gottweis, Juraj, Tomasev, Nenad, Liu, Yun, Rajkomar, Alvin, Barral, Joelle, Semturs, Christopher, Karthikesalingam, Alan, Natarajan, Vivek

  3. FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation — published result table ↗

    Vu, Tu, Iyyer, Mohit, Wang, Xuezhi, Constant, Noah, Wei, Jerry, Wei, Jason et al. · 2023

    All 11 authors

    Vu, Tu, Iyyer, Mohit, Wang, Xuezhi, Constant, Noah, Wei, Jerry, Wei, Jason, Tar, Chris, Sung, Yun-Hsuan, Zhou, Denny, Le, Quoc, Luong, Thang

Names used in source tables

PaLM (8B) · PaLM 8B