RLTasks by ulam.ai

Models / Reported model

Reported model / MODEL EVIDENCE

Llama 3-ChatQA-1.5-8B + RAG

1 benchmarks · 72 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: Llama 3-ChatQA-1.5-8B + RAG; small dots: closest models where reported

BABILong ↗48%

Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 0k context · historical v0 leaderboard ↗

71 more reported results

46% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

45% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 16k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

42% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 64k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

45% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 128k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

39% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 1M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

48% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

47% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

45% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

44% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 32k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

42% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 512k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

37% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1–QA5 average · 10M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

60% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

62% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

60% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

58% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

58% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

60% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 16k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

60% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 32k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

56% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 64k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

64% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 128k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

54% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 512k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

55% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 1M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

50% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA1 · 10M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

28% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

25% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

22% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

19% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

14% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

13% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 16k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

9% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 32k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

7% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 64k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

6% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 128k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

6% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 512k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

2% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 1M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

2% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA2 · 10M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

17% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

18% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

17% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

17% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

16% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

17% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 16k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

15% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 32k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

13% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 64k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

19% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 128k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

17% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 512k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

10% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 1M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

11% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA3 · 10M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

53% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

58% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

56% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

59% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

57% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

60% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 16k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

60% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 32k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

59% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 64k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

60% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 128k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

59% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 512k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

54% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 1M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

56% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA4 · 10M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

80% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 0k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

77% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 1k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

78% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 2k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

77% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 4k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

78% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 8k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

77% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 16k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

78% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 32k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

76% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 64k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

75% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 128k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

75% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 512k context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

76% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 1M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

67% · Llama3-ChatQA-1.5-8B + RAG · BABILong · BABILong v0 · QA5 · 10M context · historical v0 leaderboard · BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

LongAlpaca-13B →

0.0 percentile-point gap

1 of 1 shared benchmark comparisons

Llama-2-7B-32K-Instruct →Meta

5.2 percentile-point gap

1 of 1 shared benchmark comparisons

longchat-7b-v1.5-32k →

5.2 percentile-point gap

1 of 1 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard — published result table ↗
Kuratov, Yuri, Bulatov, Aydar, Anokhin, Petr, Rodkin, Ivan, Sorokin, Dmitry, Sorokin, Artyom et al. · 2024
All 7 authors
Kuratov, Yuri, Bulatov, Aydar, Anokhin, Petr, Rodkin, Ivan, Sorokin, Dmitry, Sorokin, Artyom, Burtsev, Mikhail

Names used in source tables

Llama3-ChatQA-1.5-8B + RAG