Predictive Validity Assessment on Mental Health Task-Oriented Dialogue (TOD) 160K (test)
0.818Pearson Correlation (r)UserSim-160K-it2
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UserSim-160K-it2training_stage=adversarial iteration 22025.12 | 0.818 | 0.03 | 0.318 | |
| Aggregatecomposition=combination of UserSim-160K-base, it1, it2, and it32025.12 | 0.801 | 0.06 | 0.152 | |
| UserSim-160K-it3training_stage=adversarial iteration 32025.12 | 0.781 | 0.1 | 0.148 | |
| UserSim-160K-it1training_stage=adversarial iteration 12025.12 | 0.685 | 0.7 | 0.63 | |
| Llama-3.3-70B-Instructzero-shot=true2025.12 | 0.629 | 1.6 | 0.542 | |
| Kimi-K2-Instructzero-shot=true2025.12 | 0.564 | 3.6 | 0.453 | |
| UserSim-160K-basetraining_stage=supervised fine-tuning2025.12 | 0.528 | 5.2 | 0.671 | |
| GPT-4ozero-shot=true2025.12 | 0.484 | 7.9 | 1.566 |