Agentic Tool Use on τ²-Bench Telecom
100AccuracyGPT 5.4
Evaluation results
| Method | Links | |
|---|---|---|
| GPT 5.4Deployment=Cloud baseline2026.05 | 100 | |
| Qwen3.5-27BArchitecture=Dense, # Total Params=27B, # Activated Params=27B, Reasoning Mode=REASONING2026.04 | 99.3 | |
| Claude Opus 4.6Deployment=Cloud baseline2026.05 | 89.4 | |
| Qwen3.5-122BDeployment=Local model2026.05 | 86.2 | |
| Best LocalConfig=Oracle local routing frontier2026.05 | 86.2 | |
| Gemini 3.1 ProDeployment=Cloud baseline2026.05 | 85 | |
| Qwen3.5-27BDeployment=Local model2026.05 | 84.9 | |
| Qwen3.5-35BDeployment=Local model2026.05 | 83.1 | |
| Gemma4-26BDeployment=Local model2026.05 | 78.5 | |
| Qwen3.5-9BDeployment=Local model2026.05 | 75.3 | |
| GPT-5 miniReasoning Mode=REASONING: HIGH2026.04 | 74.1 | |
| K-EXAONE-236B-A23BArchitecture=MoE, # Total Params=236B, # Activated Params=23B, Reasoning Mode=REASONING2026.04 | 73.5 | |
| EXAONE 4.5 33BArchitecture=Dense, # Total Params=33B, # Activated Params=32B, Reasoning Mode=REASONING2026.04 | 73 | |
| Nemotron-Super-120BDeployment=Local model2026.05 | 68.3 | |
| Gemma4-E4BDeployment=Local model2026.05 | 61.3 | |
| Qwen3-VL-235B-A22BArchitecture=MoE, # Total Params=236B, # Activated Params=22B, Reasoning Mode=Thinking2026.04 | 44.7 | |
| Granite 3.3 8BDeployment=Local model2026.05 | 5.3 | |
| Granite 4.0 H-SmallDeployment=Local model2026.05 | 0 |