BABILong

Sources

1. BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack — official v0 leaderboard — published result table ↗

   Kuratov, Yuri, Bulatov, Aydar, Anokhin, Petr, Rodkin, Ivan, Sorokin, Dmitry, Sorokin, Artyom et al. · 2024

   All 7 authors

   Kuratov, Yuri, Bulatov, Aydar, Anokhin, Petr, Rodkin, Ivan, Sorokin, Dmitry, Sorokin, Artyom, Burtsev, Mikhail

   Dates & evaluation details

   BABILong · BABILong v0 · QA1–QA5 average · 0k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 1k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 2k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 4k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 8k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 16k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 32k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 64k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 128k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 512k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 1M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1–QA5 average · 10M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 0k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 1k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 2k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 4k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 8k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 16k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 32k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 64k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 128k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 512k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 1M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA1 · 10M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 0k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 1k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 2k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 4k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 8k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 16k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 32k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 64k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 128k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 512k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 1M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA2 · 10M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 0k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 1k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 2k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 4k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 8k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 16k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 32k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 64k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 128k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 512k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 1M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA3 · 10M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 0k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 1k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 2k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 4k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 8k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 16k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 32k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 64k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 128k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 512k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 1M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA4 · 10M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 0k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 1k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 2k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 4k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 8k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 16k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 32k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 64k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 128k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 512k context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 1M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   BABILong · BABILong v0 · QA5 · 10M context · historical v0 leaderboard · 2026-10-27

   Published model comparison, 2024. BABILong · historical v0 leaderboard. Published accuracy on the source task and context length. The avg row is the author’s QA1–QA5 average; single tasks, context lengths and this historical v0 snapshot remain separate measures. Missing lengths are omitted. The leaderboard’s newer consolidated result file is not included in this historical extract. Published result table; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

TRACE THE SCORE · Per-model notes use published breakdowns where available.