Utterance-level User Simulation on Chinese User Simulation Dataset
69.92AI ProbabilityUserLM
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| UserLMModel=UserLM2026.04 | 69.92 | 58.88 | 55.38 | 51.11 | 52.89 | 47.42 | 56.55 | |
| GPT-4oModel=GPT-4o2026.04 | 45.5 | 71.81 | 62.79 | 88.91 | 92.21 | 83.04 | 92.14 | |
| Qwen3-8BModel=Qwen3-8B2026.04 | 44.45 | 73.07 | 64.69 | 86.44 | 91.02 | 78.88 | 89.76 | |
| USPModel=USP2026.04 | 43.14 | 66.44 | 59.25 | 63.84 | 68.74 | 57.79 | 73.14 | |
| Muse (w/o RL)Reinforcement Learning=Excluded2026.04 | 37.98 | 76.06 | 64.8 | 90.46 | 95.91 | 85.3 | 93.89 | |
| MuseReinforcement Learning=Included2026.04 | 31.18 | 75.34 | 64.89 | 91.96 | 97.76 | 87.63 | 96.2 |