LLM Agent Task Completion on τ-Bench (Retail & Airline Average)
18.1Success Rate (%)DeepSeek-V3.1
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| DeepSeek-V3.1Clarify Strategy=DeepSeek-V3.12026.06 | 18.1 | 21 | 3.9 | 24.9 | |
| None (Full User Intent)Clarify Strategy=None (Full User Intent)2026.06 | 18 | 22.3 | 0 | 22.3 | |
| Ours (Qwen3-1.7B-trained)Clarify Strategy=Ours (Qwen3-1.7B-trained)2026.06 | 17.8 | 23.8 | 1.3 | 25.1 | |
| Qwen3-30BClarify Strategy=Qwen3-30B2026.06 | 17.4 | 21 | 3.8 | 24.8 | |
| DeepSeek-R1Clarify Strategy=DeepSeek-R12026.06 | 16.9 | 20.5 | 3.9 | 24.4 | |
| Qwen3-14BClarify Strategy=Qwen3-14B2026.06 | 16.3 | 24.2 | 5.1 | 29.3 | |
| NoneClarify Strategy=None2026.06 | 15.4 | 24.2 | 0 | 24.2 | |
| Qwen3-8BClarify Strategy=Qwen3-8B2026.06 | 11.9 | 23.1 | 2.7 | 25.8 |