Terminal-bench
Benchmarks
TaskDatasetSOTA resultTrendResults
Terminal-bench 2.0
64.7Pass@1
90
Terminal-Bench 2
83.14SSA Pass@1 Rate
34
Terminal-Bench 2.0
59.3Success Rate (SR)
32
Terminal-Bench 2.0
82TB 2.0 Score
27
Terminal-Bench 2.0 (full)
31.9Pass@1
19
Terminal-Bench 2.0
64.7Accuracy
19
Terminal-Bench (dev)
37.4TB-dev Score
18
Terminal Bench 2.0
59.1Pass@1
18
Terminal-Bench v1
47.5Success Rate
17
Terminal-bench 1.0
51Pass@1
17
Terminal-Bench (TB2) 2.0
36Resolution Rate
16
Terminal-Bench 2.0
59.6Success Rate
15
Terminal-Bench 2
49.6Score
13
Terminal-Bench 2 (out-of-distribution)
34.12Task Success Rate
12
Terminal-Bench 1.0
51Accuracy
12
Terminal-Bench 2.0
27.4Accuracy
12
Terminal-Bench
45Accuracy
12
Terminal-Bench Hard
57.6Score
12
Terminal Bench Pro
37Pass@1
11
Terminal Bench 1.0
33.44Pass@1
11
Terminal Bench 2.0
48.31Pass@1
11
Terminal-Bench 2.0
59.3Score
11
Terminal-Bench 2.0
65.2Success Rate (SR)
10
Terminal-Bench (TB) (test)
48.75Success Rate
10
Terminal-Bench 1.1
43Resolved Rate
9