Agentic Task Completion on τ-bench Average
40Success RateIG-clarifier
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| IG-clarifierAgent Model=DeepSeek-R1, Clarifier Model=Ours2026.06 | 40 | 21.8 | 0.5 | 22.3 | |
| IG-clarifierAgent Model=DeepSeek-V3.2, Clarifier Model=Ours2026.06 | 39.7 | 23.1 | 0.7 | 23.8 | |
| Qwen3-8BAgent Model=DeepSeek-V3.2, Clarifier Model=Qwen3-8B2026.06 | 39.6 | 23.3 | 2.7 | 26 | |
| NoneAgent Model=DeepSeek-R1, Clarifier Model=None2026.06 | 35.7 | 18.1 | 0 | 18.1 | |
| Qwen3-8BAgent Model=DeepSeek-R1, Clarifier Model=Qwen3-8B2026.06 | 34 | 11.5 | 1 | 12.5 | |
| NoneAgent Model=DeepSeek-V3.2, Clarifier Model=None2026.06 | 33.3 | 23.5 | 0 | 23.5 | |
| IG-clarifierAgent Model=Agent Avg., Clarifier Model=Ours2026.06 | 27.3 | 23.7 | 1.2 | 24.9 | |
| NoneAgent Model=Agent Avg., Clarifier Model=None2026.06 | 23.6 | 24.6 | 0 | 24.6 | |
| Qwen3-8BAgent Model=Agent Avg., Clarifier Model=Qwen3-8B2026.06 | 23 | 22.8 | 2.6 | 25.6 | |
| IG-clarifierAgent Model=Qwen3-32B, Clarifier Model=Ours2026.06 | 21.2 | 16.9 | 2.6 | 19.5 | |
| NoneAgent Model=Qwen3-32B, Clarifier Model=None2026.06 | 18.7 | 24 | 0 | 24 | |
| IG-clarifierAgent Model=Qwen3-8B, Clarifier Model=Ours2026.06 | 17.8 | 23.8 | 1.3 | 25.1 | |
| IG-clarifierAgent Model=GLM-4-32B, Clarifier Model=Ours2026.06 | 16.5 | 32.9 | 1 | 33.9 | |
| Qwen3-8BAgent Model=Qwen3-32B, Clarifier Model=Qwen3-8B2026.06 | 15.9 | 20.4 | 5.1 | 25.5 | |
| NoneAgent Model=Qwen3-8B, Clarifier Model=None2026.06 | 15.4 | 24.2 | 0 | 24.2 | |
| NoneAgent Model=GLM-4-32B, Clarifier Model=None2026.06 | 15 | 33.3 | 0 | 33.3 | |
| Qwen3-8BAgent Model=GLM-4-32B, Clarifier Model=Qwen3-8B2026.06 | 13.5 | 35.6 | 1.5 | 37.1 | |
| Qwen3-8BAgent Model=Qwen3-8B, Clarifier Model=Qwen3-8B2026.06 | 11.9 | 23.1 | 2.7 | 25.8 |