Tool-use on Tau-Bench
67.5TAU-AIR ScoreClaude-Sonnet-4
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude-Sonnet-4Category=Proprietary Models2026.02 | 67.5 | 54 | 47.4 | 55.91 | |
| Gemini-2.5-proCategory=Proprietary Models2026.02 | 67.5 | 56 | 27.2 | 44.99 | |
| DeepSeek-V3.2Category=Open-Source Foundation Models2026.02 | 63.8 | 74.12 | 96.2 | 69.75 | |
| GPT-5Category=Proprietary Models2026.02 | 58 | 77.2 | 95.8 | 68.69 | |
| GPT-4.1Category=Proprietary Models2026.02 | 56 | 74 | 34 | 50.72 | |
| SimiaCategory=Tool-Learning Methods, Training Samples=50002026.02 | 52 | 58.77 | 47.15 | 45.29 | |
| TDScalingBase Model=Qwen3-Coder-30B-A3B-Instruct, Training Samples=50002026.02 | 44 | 64.69 | 60.75 | 52.47 | |
| Qwen3-Coder-480B-A35B-InstructCategory=Open-Source Foundation Models2026.02 | 41 | 63.82 | 66.67 | 51.85 | |
| TDScalingBase Model=Qwen3-Coder-30B-A3B-Instruct, Training Samples=5002026.02 | 40 | 63.38 | 55.92 | 48.99 | |
| TDScalingBase Model=Qwen3-30B-A3B-Instruct, Training Samples=5002026.02 | 39 | 58.55 | 31.8 | 41.5 | |
| Qwen3-Coder-30B-A3B-InstructCategory=Base Model2026.02 | 36.5 | 58.55 | 41.45 | 41.48 | |
| TOUCANCategory=Tool-Learning Methods, Training Samples=50002026.02 | 33.5 | 56.36 | 56.8 | 45.92 | |
| APIGen-MTCategory=Tool-Learning Methods, Training Samples=50002026.02 | 33 | 60.75 | 50.22 | 42.81 | |
| Qwen3-30B-A3B-InstructCategory=Base Model2026.02 | 30.5 | 53.29 | 21.93 | 34.74 |