Terminal Command Execution on TerminalBench
98.1Success RateLLM-only
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLM-onlyBackbones=Averaged over Gemma-9B, LLaMA-3.1-8B, Qwen2.5-7B, Qwen2.5-14B2026.05 | 98.1 | 100 | |
| Oracle RouterBackbones=Averaged over Gemma-9B, LLaMA-3.1-8B, Qwen2.5-7B, Qwen2.5-14B2026.05 | 97.8 | 18.4 | |
| Heuristic RouterBackbones=Averaged over Gemma-9B, LLaMA-3.1-8B, Qwen2.5-7B, Qwen2.5-14B2026.05 | 95 | 66 | |
| R2VBackbones=Averaged over Gemma-9B, LLaMA-3.1-8B, Qwen2.5-7B, Qwen2.5-14B, alpha=0.20, epsilon=0.102026.05 | 93.3 | 33.9 | |
| Entropy RouterBackbones=Averaged over Gemma-9B, LLaMA-3.1-8B, Qwen2.5-7B, Qwen2.5-14B2026.05 | 87.4 | 4.1 | |
| SLM-onlyBackbones=Averaged over Gemma-9B, LLaMA-3.1-8B, Qwen2.5-7B, Qwen2.5-14B2026.05 | 81.1 | 0 |