User Simulation on User Simulation Dataset Session-level (test)
95.21Role ScoreUserLM-R1-32B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| UserLM-R1-32BModel Size=32B2026.01 | 95.21 | 46.08 | 57.46 | 73.49 | |
| DeepSeek-R12026.01 | 94.29 | 58.75 | 46.92 | 73.56 | |
| UserLM-R1-8BModel Size=8B2026.01 | 92.25 | 32.71 | 45.96 | 65.79 | |
| Gemini-2.5-Flash2026.01 | 92 | 54.87 | 40.79 | 69.92 | |
| Qwen3-32BModel Size=32B, Chain-of-thought generation=true2026.01 | 89.29 | 37.71 | 41.46 | 64.44 | |
| Qwen3-32BModel Size=32B, Chain-of-thought generation=false2026.01 | 87.67 | 38.92 | 37.96 | 63.06 | |
| Xingchen2026.01 | 86.09 | 56.34 | 44.7 | 68.31 | |
| Qwen3-8BModel Size=8B, Chain-of-thought generation=false2026.01 | 85.57 | 29.85 | 34.21 | 58.8 | |
| Qwen3-8BModel Size=8B, Chain-of-thought generation=true2026.01 | 85.24 | 30.25 | 37.88 | 59.65 | |
| Doubao-Seed-1.62026.01 | 84.58 | 47.54 | 39.71 | 64.1 | |
| CharacterGLM2026.01 | 45.58 | 13.87 | 14.5 | 29.88 |