Linguistic Diversity Analysis on Real-world vs. Simulated User Conversations
7.65EntropyKimi-K2-Instruct
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Kimi-K2-InstructMode=zero-shot2025.12 | 7.65 | 0.416 | 0.851 | |
| GPT-4oMode=zero-shot2025.12 | 7.12 | 0.408 | 0.837 | |
| Llama-3.3-70B-InstructMode=zero-shot2025.12 | 7.02 | 0.268 | 0.651 | |
| Real UsersType=Human Baseline2025.12 | 6.6 | 0.483 | 0.861 | |
| UserSim-160K-it2Training Data Size=160K, Training Stage=iteration 22025.12 | 6.52 | 0.439 | 0.821 | |
| UserSim-160K-it3Training Data Size=160K, Training Stage=iteration 32025.12 | 6.51 | 0.461 | 0.841 | |
| UserSim-160K-it1Training Data Size=160K, Training Stage=iteration 12025.12 | 6.38 | 0.471 | 0.839 | |
| UserSim-20K-baseTraining Data Size=20K, Training Stage=base2025.12 | 6.23 | 0.422 | 0.782 | |
| UserSim-160K-baseTraining Data Size=160K, Training Stage=base2025.12 | 6.18 | 0.382 | 0.728 | |
| UserSim-80K-baseTraining Data Size=80K, Training Stage=base2025.12 | 6.15 | 0.409 | 0.759 | |
| UserSim-10K-baseTraining Data Size=10K, Training Stage=base2025.12 | 6.14 | 0.406 | 0.756 | |
| UserSim-40K-baseTraining Data Size=40K, Training Stage=base2025.12 | 6.14 | 0.43 | 0.782 |