Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

TerminalBench dataset and benchmarks · SOTA2 Research

Research Datasets

TerminalBench

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Terminal-related CLI agent task

TerminalBench 1.0

54.4Accuracy

51

Terminal-related CLI agent task

TerminalBench 2.0

57.8Accuracy

29

Terminal Agentic Trajectory Generation

TerminalBench 2.0

57.8Score

29

Terminal Agent Tasks

TerminalBench 2.0

57.8Success Score

25

Terminal Agentic Trajectory Generation

TerminalBench 1.0

56.25Score

23

TerminalBench 2

81.8Pass Rate

17

Prefix-risk ranking

TerminalBench (held-out)

0.557AUPRC

11

Vulnerability Discovery

TerminalBench 2 snapshot 2026-04-17

84.3Score (%)

11

Multimodal Agentic Tool Use

TerminalBench-O

24Pass Rate

9

Code Generation

TerminalBench 2

39.3Pass@3

9

TerminalBench

0.338Accuracy

7

Terminal Command Execution

TerminalBench

98.1Success Rate

6

Terminal-based Tool Use

TerminalBench (TBench)

57.2Pass@1

5

Ranking Preservation

TerminalBench (test)

0.988Mean Spearman Rho

5

Terminal Task Execution

TerminalBench 2.0 (test)

35.2Test Accuracy

4

Terminal Agentic Trajectory Generation

TerminalBench

45Pass@8

4

TerminalBench 1.0

0.344Score

2

TerminalBench 2.0

0.243Score

2

Popular tasks

Agentic Coding Terminal Agentic Trajectory Generation Terminal-related CLI agent task Code Generation Terminal Task Execution

Follow for updates

Join SOTA2 Contact