Terminal Task Completion on Terminal-bench 2.0
64.7Pass@1Gemini 3 Pro
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini 3 ProOpenSource=false, Agent Framework=Best with Any Agent2026.02 | 64.7 | — | — | — | — | |
| GPT 5.2OpenSource=false, Agent Framework=Best with Any Agent2026.02 | 62.9 | — | — | — | — | |
| Claude Opus 4.5OpenSource=false, Agent Framework=Best with Any Agent2026.02 | 57.8 | — | — | — | — | |
| AORCHESTRAModel Setup=Gemini-3-Flash2026.02 | 52.86 | 57.14 | 71.62 | — | — | |
| GLM 5Model Size=744B2026.06 | 52.4 | — | — | — | — | |
| Claude Opus 4.5OpenSource=false, Agent Framework=OpenHands2026.02 | 51.9 | — | — | — | — | |
| MiniMax M2.7Model Size=230B2026.06 | 45.1 | — | — | — | — | |
| GPT-5OpenSource=false, Agent Framework=OpenHands2026.02 | 43.8 | — | — | — | — | |
| Kimi K2.5Model Size=1T2026.06 | 43.2 | — | — | — | — | |
| Claude Sonnet 4.5OpenSource=false, Agent Framework=Best with Any Agent2026.02 | 42.8 | — | — | — | — | |
| TMAX-27BModel Size=27B2026.06 | 42.7 | — | — | — | — | |
| Claude Sonnet 4.5OpenSource=false, Agent Framework=OpenHands2026.02 | 42.6 | — | — | — | — | |
| DeepSeek-v3.2Model Size=671B2026.06 | 39.6 | — | — | — | — | |
| Qwen 3.6Model Size=27B2026.06 | 39.6 | — | — | — | — | |
| Claude Opus 4.1OpenSource=false, Agent Framework=Best with Any Agent2026.02 | 38 | — | — | — | — | |
| Claude Opus 4.1OpenSource=false, Agent Framework=OpenHands2026.02 | 36.9 | — | — | — | — | |
| Minimax-M2.1OpenSource=true, Agent Framework=Best with Any Agent2026.02 | 36.6 | — | — | — | — | |
| AORCHESTRAModel Setup=Claude-4.5-haiku2026.02 | 35.71 | 45.71 | 55.44 | — | — | |
| Mini-SWEModel Setup=Gemini-3-Flash2026.02 | 34.29 | 50 | 49.49 | — | — | |
| Claude CodeModel Setup=Claude-4.5-haiku2026.02 | 34.29 | 45.71 | 29.65 | — | — | |
| Claude CodeModel Setup=Gemini-3-Flash2026.02 | 32.86 | 48.57 | 27.43 | — | — | |
| Gemini 2.5 ProOpenSource=false, Agent Framework=Best with Any Agent2026.02 | 32.6 | — | — | — | — | |
| GPT-5-miniModel Size=?2026.06 | 31.9 | — | — | — | — | |
| OpenHandsModel Setup=Gemini-3-Flash2026.02 | 31.43 | 51.43 | 48.49 | — | — | |
| AORCHESTRAModel Setup=DeepSeek-V3.22026.02 | 31.43 | 42.86 | 58.43 | — | — | |
| LiberCoder-235B-A22BOpenSource=true, Agent Framework=OpenHands2026.02 | 31 | — | — | — | — | |
| LiberCoderModel Size=235B2026.06 | 31 | — | — | — | — | |
| Minimax-M2OpenSource=true, Agent Framework=Best with Any Agent2026.02 | 30 | — | — | — | — | |
| Mini-SWEModel Setup=DeepSeek-V3.22026.02 | 30 | 48.57 | 54.76 | — | — | |
| Claude Haiku 4.5OpenSource=false, Agent Framework=Best with Any Agent2026.02 | 29.8 | — | — | — | — | |
| Claude Haiku 4.5Model Size=?2026.06 | 29.8 | — | — | — | — | |
| MTL (I)Model Backbone=DeepSeek V3.22026.04 | 28.8 | — | — | — | — | |
| ReActModel Setup=Gemini-3-Flash2026.02 | 28.57 | 47.14 | 47.22 | — | — | |
| ZeroShotModel Backbone=DeepSeek V3.22026.04 | 28.5 | — | — | — | — | |
| Kimi-K2-InstructOpenSource=true, Agent Framework=Best with Any Agent2026.02 | 27.8 | — | — | — | — | |
| Nemotron-TerminalModel Size=32B2026.06 | 27.4 | — | — | — | — | |
| Grok 4OpenSource=false, Agent Framework=OpenHands2026.02 | 27.2 | — | — | — | — | |
| Grok 4OpenSource=false, Agent Framework=Best with Any Agent2026.02 | 27.2 | — | — | — | — | |
| Qwen3-Coder-480B-A35B-InstructOpenSource=true, Agent Framework=Best with Any Agent2026.02 | 27.2 | — | — | — | — | |
| TMAX-9BModel Size=9B2026.06 | 27.2 | — | — | — | — | |
| Kimi-K2-InstructOpenSource=true, Agent Framework=OpenHands2026.02 | 26.7 | — | — | — | — | |
| Qwen3-Coder-480B-A35B-InstructOpenSource=true, Agent Framework=OpenHands2026.02 | 25.4 | — | — | — | — | |
| GLM-4.6OpenSource=true, Agent Framework=Best with Any Agent2026.02 | 24.5 | — | — | — | — | |
| MTL (W)Model Backbone=GPT-5-mini2026.04 | 24.3 | — | — | — | — | |
| Mini-SWEModel Setup=Claude-4.5-haiku2026.02 | 24.29 | 28.57 | 36.3 | — | — | |
| TerminalTraj-32BScaffold=Terminus-22026.05 | 23.88 | — | — | — | 37.08 | |
| MTL (S)Model Backbone=GPT-5-mini2026.04 | 22.8 | — | — | — | — | |
| MTL (I)Model Backbone=Qwen3-Coder-480B-A35B-Instruct2026.04 | 22.8 | — | — | — | — | |
| TerminalTrajModel Size=32B2026.06 | 22 | — | — | — | — | |
| OpenHandsModel Setup=DeepSeek-V3.22026.02 | 21.43 | 35.71 | 48.35 | — | — | |
| Nemotron-Terminal-32BScaffold=Terminus-22026.05 | 21.35 | — | — | — | 35.96 | |
| MTL (I)Model Backbone=GPT-5-mini2026.04 | 21.3 | — | — | — | — | |
| Qwen 3.5Model Size=9B2026.06 | 21.1 | — | — | — | — | |
| ZeroShotModel Backbone=Qwen3-Coder-480B-A35B-Instruct2026.04 | 21 | — | — | — | — | |
| ZeroShotModel Backbone=GPT-5-mini2026.04 | 20.6 | — | — | — | — | |
| Nemotron-TerminalModel Size=14B2026.06 | 20.2 | — | — | — | — | |
| ReActModel Setup=DeepSeek-V3.22026.02 | 20 | 32.86 | 38.23 | — | — | |
| ReActModel Setup=Claude-4.5-haiku2026.02 | 20 | 37.14 | 43.62 | — | — | |
| LiberCoder-32BOpenSource=true, Agent Framework=OpenHands2026.02 | 19.5 | — | — | — | — | |
| MTL (T)Model Backbone=GPT-5-mini2026.04 | 19.5 | — | — | — | — | |
| LiberCoderModel Size=32B2026.06 | 19.5 | — | — | — | — | |
| TermiGenModel Size=32B2026.06 | 19.3 | — | — | — | — | |
| TerminalTrajModel Size=14B2026.06 | 19.1 | — | — | — | — | |
| TMAX-4BModel Size=4B2026.06 | 18.9 | — | — | — | — | |
| GPT-OSS-120BOpenSource=true, Agent Framework=Best with Any Agent2026.02 | 18.7 | — | — | — | — | |
| GPT-OSSModel Size=120B2026.06 | 18.7 | — | — | — | — | |
| Qwen3-32B-Nex-N1Scaffold=OpenHands2026.05 | 18.54 | — | — | — | 26.97 | |
| LiteCoder-Terminal-32b-sftScaffold=Terminus-22026.05 | 18.54 | — | — | — | 30.34 | |
| Qwen3-235B-A22B-InstructOpenSource=true, Agent Framework=OpenHands2026.02 | 18.1 | — | — | — | — | |
| Qwen 3.5Model Size=4B2026.06 | 16.6 | — | — | — | — | |
| Gemini 2.5 ProOpenSource=false, Agent Framework=OpenHands2026.02 | 16.4 | — | — | — | — | |
| Claude Haiku 4.5OpenSource=false, Agent Framework=OpenHands2026.02 | 13.9 | — | — | — | — | |
| Nemotron-TerminalModel Size=8B2026.06 | 13 | — | — | — | — | |
| Qwen3-Coder-30B-A3B-InstructOpenSource=true, Agent Framework=OpenHands2026.02 | 12.9 | — | — | — | — | |
| Qwen3-Coder-30B-A3B-InstructOpenSource=true, Agent Framework=Best with Any Agent2026.02 | 12.9 | — | — | — | — | |
| OpenHandsModel Setup=Claude-4.5-haiku2026.02 | 12.85 | 25.71 | 45.13 | — | — | |
| Qwen3-30B-A3B-Nex-N1Scaffold=OpenHands2026.05 | 12.36 | — | — | — | 23.6 | |
| LiteCoder-Terminal-30b-a3b-sftScaffold=Terminus-22026.05 | 12.36 | — | — | — | 23.6 | |
| GPT-5-nanoModel Size=?2026.06 | 11.5 | — | — | — | — | |
| TerminalTrajModel Size=7B2026.06 | 10.1 | — | — | — | — | |
| EndlessTerminalsModel Size=8B, Training Mode=OT SFT + RL2026.06 | 6.7 | — | — | — | — | |
| Qwen3-32BOpenSource=true, Agent Framework=OpenHands2026.02 | 5.7 | — | — | — | — | |
| Qwen3-30B-A3B-InstructScaffold=Terminus-22026.05 | 5.34 | — | — | — | 11.24 | |
| OpenThinker-Agent-v1Model Size=8B2026.06 | 4.9 | — | — | — | — | |
| LiteCoder-Terminal-4b-sftScaffold=Terminus-22026.05 | 4.78 | — | — | — | 10.11 | |
| Qwen2.5-Coder-32B-InstructScaffold=Terminus-22026.05 | 4.49 | — | — | — | 8.99 | |
| OpenThinker-Agent-v1Scaffold=Terminus-22026.05 | 4.49 | — | — | — | 10.1 | |
| TMAX-2BModel Size=2B2026.06 | 2.9 | — | — | — | — | |
| Qwen 3.5Model Size=2B2026.06 | 2.3 | — | — | — | — | |
| Qwen3-4B-InstructScaffold=Terminus-22026.05 | 1.12 | — | — | — | 3.37 | |
| BConfiguration=5 native flags, Backbone=gpt-5.4-nano2026.04 | — | — | — | 17 | — | |
| BaselineConfiguration=no TURBOQUANT, Backbone=gpt-5.4-nano2026.04 | — | — | — | 15 | — | |
| DConfiguration=all-on, 8 flags, Backbone=gpt-5.4-nano2026.04 | — | — | — | 12 | — | |
| HARBORConfiguration=2 flags, Backbone=gpt-5.4-nano2026.04 | — | — | — | 17 | — | |
| OracleConfiguration=union over configs, Backbone=gpt-5.4-nano2026.04 | — | — | — | 81 | — |