Stateful service dialogue on tau2-bench Telecom
85.1Task Completion Score (TCS)DeepSeek-V3.2
Evaluation results
| Method | Links | |
|---|---|---|
| DeepSeek-V3.2Average Steps (Stp)=34.6, Input Tokens (Tok In)=246.9, Output Tokens (Tok Out)=2.8, Execution Time=175.22026.05 | 85.1 | |
| GPT-5-miniAverage Steps (Stp)=32.9, Input Tokens (Tok In)=296.4, Output Tokens (Tok Out)=41.4, Execution Time=1479.32026.05 | 79.9 | |
| Qwen3-235B-A22BAverage Steps (Stp)=24.8, Input Tokens (Tok In)=180.2, Output Tokens (Tok Out)=1.8, Execution Time=200.42026.05 | 67.5 | |
| Gemini-3-flashAverage Steps (Stp)=23.2, Input Tokens (Tok In)=197.9, Output Tokens (Tok Out)=6.5, Execution Time=404.92026.05 | 35.1 |