What it measures

Virtual interactive environment where agents formulate hypotheses, design controlled experiments, gather data, and discover scientific laws.

Reading the measurement

Ask whether the evaluation rewards recalling a fact, constructing a model, running an experiment or supporting a discovery. Those tasks require different evidence. For research claims, inspect reproducibility, uncertainty and what was actually checked.

Published evaluation

Published difficulty-level mean over discovery tasks and parametric variants. GPT-4o is used by all three agent scaffolds. Procedural credit, task completion and explanatory knowledge are distinct metrics.

Where this fits in an evaluation suite

A correct multiple-choice answer and a supported scientific contribution demand different evidence. Check whether the score rewards a proposal, an executed experiment, or a completed research target.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Published difficulty-level mean over discovery tasks and parametric variants. GPT-4o is used by all three agent scaffolds. Procedural credit, task completion and explanatory knowledge are distinct metrics.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.

PRIMARY EVIDENCE & FURTHER READING

  1. DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents — published result table ↗

    Jansen, Peter, Côté, Marc-Alexandre, Khot, Tushar, Bransom, Erin, Mishra, Bhavana Dalvi, Majumder, Bodhisattwa Prasad et al. · 2024

    All 8 authors

    Jansen, Peter, Côté, Marc-Alexandre, Khot, Tushar, Bransom, Erin, Mishra, Bhavana Dalvi, Majumder, Bodhisattwa Prasad, Tafjord, Oyvind, Clark, Peter

Reviewed 2026-10-27. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.