Agent Task Completion on τ-bench airline
19Success RateQwen3-30B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-30BClarify Strategy=Qwen3-30B2026.06 | 19 | — | — | — | — | — | |
| DeepSeek-R1Clarify Strategy=DeepSeek-R12026.06 | 19 | — | — | — | — | — | |
| Ours (Qwen3-1.7B-trained)Clarify Strategy=Ours (Qwen3-1.7B-trained)2026.06 | 17.3 | — | — | — | — | — | |
| DeepSeek-V3.1Clarify Strategy=DeepSeek-V3.12026.06 | 17 | — | — | — | — | — | |
| Qwen3-14BClarify Strategy=Qwen3-14B2026.06 | 16 | — | — | — | — | — | |
| None (Full User Intent)Clarify Strategy=None (Full User Intent)2026.06 | 16 | — | — | — | — | — | |
| NoneClarify Strategy=None2026.06 | 14.3 | — | — | — | — | — | |
| Qwen3-8BClarify Strategy=Qwen3-8B2026.06 | 8 | — | — | — | — | — | |
| FAMABackbone=Qwen3-14B2026.04 | — | 36.8 | 23.79 | 18.2 | 16.6 | 16 | |
| ReActBackbone=Qwen3-14B2026.04 | — | 18.4 | 12 | 9.4 | 8.4 | 8 | |
| SRBackbone=Qwen3-14B2026.04 | — | 21.2 | 15.2 | 13.3 | 12.4 | 12 |