Terminal-bench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Terminal Bench
32.5Resolve Rate
4
Terminal Bench 2.0
60Primary Score
3
Terminal-Bench 2.0
40.1Accuracy
3
Terminal Bench Core 2.0
37.5Success Rate
3
Terminal Bench hard
26.8Success Rate
3
Terminal-Bench 2.0 (held-out agent data)
0.933AUC-ROC
3
Terminal-Bench
1.5Mean Skills Retrieved per Task
3
Terminal Bench Hard English
9.9Score
3
Terminal Bench English 1.0
21.8Score
3
Terminal-Bench 2.0
25.39Cost (cents)
2
Terminal Bench (held-out)
75.2Trajectory Pass Rate
2
Terminal-Bench full 80-task old-core set 1.1
42Resolution Rate
2
Terminal-Bench 1.1 (test)
39Resolution Rate
2
Terminal Bench medium-difficulty 2
52.7Accuracy
2
Terminal-Bench Core 1.1
34.17Accuracy
2
Terminal-Bench 2.0 (test)
—Primary metric
0