RLTasksby ulam.ai
Models / Reported model

Reported model / MODEL EVIDENCE

Llama 3

2 benchmarks · 13 observations · 2 sources

Scores at a glance

2 benchmarks · large dot: Llama 3; small dots: closest models where reported

AppWorld ↗20.8%
7.148.8

LLaMA3 · AppWorld · Test-N · Task goal completion ↗

11 more reported results

8.9% · LLaMA3 · AppWorld · Test-N · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

24.4% · LLaMA3 · AppWorld · Test-N · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

8.9% · LLaMA3 · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

1.8% · LLaMA3 · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

17.9% · LLaMA3 · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

3.4% · LLaMA3 · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

2.4% · LLaMA3 · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

7% · LLaMA3 · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

0% · LLaMA3 · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

0.7% · LLaMA3 · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

4.3% · LLaMA3 · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

WorkArena & WorkArena++ ↗17.9%
6.142.7

Llama3 · WorkArena & WorkArena++ · WorkArena L1 · success rate · BrowserGym · tuned agent configurations ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

There are not enough shared scored settings to identify close comparisons reliably.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? — published result table ↗

    Drouin, Alexandre, Gasse, Maxime, Caccia, Massimo, Laradji, Issam H., Del Verme, Manuel, Marty, Tom et al. · 2024

    All 12 authors

    Drouin, Alexandre, Gasse, Maxime, Caccia, Massimo, Laradji, Issam H., Del Verme, Manuel, Marty, Tom, Boisvert, Léo, Thakkar, Megh, Cappart, Quentin, Vazquez, David, Chapados, Nicolas, Lacoste, Alexandre

  2. AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents — published result table ↗

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li et al. · 2024

    All 9 authors

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, Niranjan Balasubramanian

Names used in source tables

LLaMA3 · Llama3