Tool-use Task Completion on Tau-Bench Airline v2 (test)
70Pass RateClaude Sonnet 4.5
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Claude Sonnet 4.52026.04 | 70 | — | |
| Qwen3-30B-A3B + IRCSize=30.5B2026.04 | 69.5 | 11.5 | |
| Qwen3-30B-A3B + MT-GRPOSize=30.5B2026.04 | 68 | 10 | |
| Qwen3.5-4B + IRCSize=4B2026.04 | 66.7 | 2.9 | |
| Qwen3.5-4B + MT-GRPOSize=4B2026.04 | 64.6 | 0.8 | |
| Qwen3.5-4B (base)Size=4B2026.04 | 63.8 | — | |
| Qwen3-30B-A3B (base)Size=30.5B2026.04 | 58 | — | |
| GPT-4.12026.04 | 49.4 | — | |
| GPT-4o2026.04 | 42.8 | — | |
| Claude 3.5 Haiku2026.04 | 22.8 | — | |
| GPT-4.1 nano2026.04 | 14 | — |