Agentic Terminal Tasks on Terminal-Bench (test)
48.75Success RateClaude Opus 4.5
Evaluation results
| Method | Links | |
|---|---|---|
| Claude Opus 4.5Base Model=Claude Opus 4.52026.02 | 48.75 | |
| Claude Sonnet 4.5Base Model=Claude Sonnet 4.52026.02 | 45 | |
| MiniMax-M2Base Model=MiniMax-M22026.02 | 37.5 | |
| Qwen 3 Coder 480BBase Model=Qwen 3 Coder 480B2026.02 | 36.25 | |
| Claude Sonnet 4Base Model=Claude Sonnet 42026.02 | 33.75 | |
| Qwen3-32B + CVE-Factory (3k)Fine-tuning Data Source=CVE-Factory, Training Data Scale (Trajectories)=3k, Base Model=Qwen3-32B2026.02 | 31.25 | |
| Qwen3-32B + CVE-Factory (4k)Fine-tuning Data Source=CVE-Factory, Training Data Scale (Trajectories)=4k, Base Model=Qwen3-32B2026.02 | 28.75 | |
| Qwen3-32B + SETAFine-tuning Data Source=SETA, Training Data Scale (Trajectories)=4k, Base Model=Qwen3-32B2026.02 | 27.5 | |
| Qwen3-Coder-30BBase Model=Qwen3-Coder-30B2026.02 | 13.75 | |
| Qwen3-32BBase Model=Qwen3-32B2026.02 | 12.5 |