Terminal interaction agent task on Terminal-Bench (dev)
37.4TB-dev ScoreGPT-5 Nano
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-5 NanoSource=Openthoughts, Context=Baselines2026.07 | 37.4 | |
| GLM-4.6Context=Baselines2026.07 | 29.9 | |
| Qwen3-235B-A22BSource=Openthoughts, Context=Baselines2026.07 | 26.9 | |
| OP-SHORTBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=10%, BC Data=5%, On-policy Data=5% (K = 3), RL=02026.07 | 19.1 | |
| BC+OPBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Type Detail=different tasks, Prefix Training Data=10%, BC Data=15%, On-policy Data=10% (K = 3), RL=720 tasks2026.07 | 18.3 | |
| OP-SHORTBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=10%, BC Data=2%, On-policy Data=2% (K = ∞), RL=02026.07 | 18 | |
| Pure BCBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=100%, BC Data=100%, On-policy Data=0, RL=720 tasks, Source=Openthoughts2026.07 | 17.3 | |
| Pure BCBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=100%, BC Data=100%, On-policy Data=0, RL=720 tasks2026.07 | 16.7 | |
| Pure OPBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=10%, BC Data=0%, On-policy Data=10% (K = ∞), RL=02026.07 | 16.7 | |
| OP-SHORTBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=10%, BC Data=2%, On-policy Data=2% (K = 3), RL=02026.07 | 16.1 | |
| BC+OPBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Type Detail=different tasks, Prefix Training Data=10%, BC Data=15%, On-policy Data=10% (K = 3), RL=02026.07 | 15.1 | |
| Pure OPBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=10%, BC Data=0%, On-policy Data=10% (K = 3), RL=02026.07 | 14.7 | |
| Pure BCBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=75%, BC Data=75%, On-policy Data=0, RL=02026.07 | 14.2 | |
| Pure BCBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=100%, BC Data=100%, On-policy Data=0, RL=02026.07 | 13.8 | |
| Pure BCBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=10%, BC Data=10%, On-policy Data=0, RL=02026.07 | 11.9 | |
| Pure BCBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=50%, BC Data=50%, On-policy Data=0, RL=02026.07 | 11.6 | |
| Pure BCBase Model=Qwen3-8B, Teacher Model=GLM-4.6, Prefix Training Data=25%, BC Data=25%, On-policy Data=0, RL=02026.07 | 10.7 | |
| Qwen3-8BSource=Openthoughts, Context=Baselines2026.07 | 5.7 |