What it measures
Very long-term multi-modal dialogue benchmark spanning 300+ turns and 35 sessions per persona pair.
Reading the measurement
F1 balances precision and recall. Check both: a system can miss relevant evidence or include unsupported material, and those errors have different consequences. Keep the matching rule, averaging policy and denominator fixed.
Published evaluation
Overall answer-prediction F1 over long-term conversational questions. Human results are excluded and context lengths are retained.
Where this fits in an evaluation suite
A large context window is a capacity specification. Successful retrieval and cross-document reasoning are observed capabilities. Test both at the actual lengths your workflow needs.
Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.
A reproducible comparison
Overall answer-prediction F1 over long-term conversational questions. Human results are excluded and context lengths are retained.
Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.
PRIMARY EVIDENCE & FURTHER READING
Reviewed 2026-10-26. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.