Tool-use agent evaluation on τ-bench airline
76Pass@1Ledger
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| LedgerBackbone=GLM-52026.06 | 76 | — | — | 56 | — | 64.6 | |
| LedgerBackbone=Kimi-K2.52026.06 | 74 | — | — | 52 | — | 62.3 | |
| Standard Function Calling (FC)Backbone=Kimi-K2.52026.06 | 69 | — | — | 44 | — | 54.4 | |
| Standard Function Calling (FC)Backbone=GLM-52026.06 | 66.5 | — | — | 40 | — | 51.3 | |
| LedgerBackbone=MiniMax M2.52026.06 | 63 | — | — | 40 | — | 49.9 | |
| Standard Function Calling (FC)Backbone=MiniMax M2.52026.06 | 61.5 | — | — | 36 | — | 46.2 | |
| ReActBackbone=Qwen3-32B2026.04 | 30.4 | 20 | 16.2 | 14.8 | 14 | — | |
| FAMABackbone=Qwen2.5-72B-Instruct2026.04 | 29.2 | 21.2 | 18.8 | 18 | 18 | — | |
| SRBackbone=Qwen2.5-72B-Instruct2026.04 | 28 | 20.4 | 17.4 | 15.6 | 14 | — | |
| FAMABackbone=Qwen3-32B2026.04 | 26.8 | 20 | 18.4 | 18 | 18 | — | |
| SRBackbone=Qwen3-32B2026.04 | 25.2 | 17 | 13.2 | 10.4 | 8 | — | |
| ReActBackbone=Qwen2.5-72B-Instruct2026.04 | 24.4 | 18.79 | 15.6 | 12.8 | 10 | — |