FocusBench

Reported in recent model launches

These model reports include this benchmark. Their harnesses, task subsets, scoring policies, and resource budgets can differ; follow each report’s evaluation notes before joining the numbers.

What it measures

Meta uses this evaluation to study whether an agent continues working toward a difficult objective.

How to read the result

Inspect persistence alongside correction: continuing an unproductive strategy is weaker evidence than adapting while preserving the original goal.

Limits and comparison traps

The public launch provides high-level framing rather than a complete open task and grading specification.

Published evaluation

Internal agent evaluation of multi-step tool use and file processing. Exact labels are transcribed from the launch chart. Full and standard Muse Wonder are distinct systems; all listed comparisons use maximum effort. The private task distribution limits independent reproduction.

Where this fits in an evaluation suite

A transcript that says “done” is weaker evidence than the verified final state. Track policy violations, recoveries, and invalid attempts alongside success.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Internal agent evaluation of multi-step tool use and file processing. Exact labels are transcribed from the launch chart. Full and standard Muse Wonder are distinct systems; all listed comparisons use maximum effort. The private task distribution limits independent reproduction.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.