User Simulation on Adversarial User Simulation Dataset Turn-level (test)
95Robotics ScoreCharacterGLM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CharacterGLM2026.01 | 95 | — | 33.97 | 37.61 | — | — | |
| Qwen3-8BModel Size=8B, Chain-of-thought generation=false2026.01 | 86.4 | — | 38.3 | 41.55 | — | — | |
| Xingchen2026.01 | 78.6 | — | 38.3 | 43.05 | — | — | |
| Qwen3-32BModel Size=32B, Chain-of-thought generation=false2026.01 | 66.4 | — | 42.98 | 47.64 | — | — | |
| Qwen3-8BModel Size=8B, Chain-of-thought generation=true2026.01 | 31.4 | 39.86 | 48.73 | 57.73 | 76.64 | 48.47 | |
| UserLM-R1-8BModel Size=8B2026.01 | 14.1 | 55.52 | 59.98 | 69.66 | 82.27 | 61.55 | |
| Qwen3-32BModel Size=32B, Chain-of-thought generation=true2026.01 | 12.3 | 41.82 | 52.86 | 62.39 | 80.14 | 53.38 | |
| Doubao-Seed-1.62026.01 | 9.5 | 41.91 | 52.64 | 63.8 | 80.73 | 53.75 | |
| Gemini-2.5-Flash2026.01 | 6.4 | 49.09 | 59.11 | 69.64 | 83.5 | 60.17 | |
| UserLM-R1-32BModel Size=32B2026.01 | 4.5 | 73.39 | 74.52 | 80.5 | 92.55 | 76.56 | |
| DeepSeek-R12026.01 | 4.1 | 45.57 | 57.15 | 66.92 | 82.97 | 58.01 |