Motivational Interviewing Behavior Evaluation on AnnoMI session-level
1.28R/QHQ (Human)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HQ (Human)Evaluation=Human counselors in high-quality sessions2026.06 | 1.28 | 52.1 | 51.4 | 82.4 | 53.4 | |
| MindfulMI_SFT+RLBackbone=GPT-4o2026.06 | 1.17 | 72.1 | 63.1 | 94.7 | 44.1 | |
| MindfulMI_SFT+RLBackbone=Llama3-70B2026.06 | 1.13 | 74.8 | 66.6 | 88.5 | 45.2 | |
| MindfulMI_SFTBackbone=GPT-4o2026.06 | 1.11 | 82.3 | 74.1 | 93.1 | 43.2 | |
| MindfulMI_SFTBackbone=Llama3-70B2026.06 | 1.04 | 87.8 | 75.7 | 87.1 | 41.8 | |
| CAMIBackbone=Llama3-70B2026.06 | 0.83 | 92.3 | 77.1 | 88.3 | 38.4 | |
| CAMIBackbone=GPT-4o2026.06 | 0.76 | 97.9 | 78.9 | 95.7 | 39.8 | |
| DIIRBackbone=Llama3-70B2026.06 | 0.73 | 91.2 | 80.1 | 86.2 | 45 | |
| DIIRBackbone=GPT-4o2026.06 | 0.47 | 97.3 | 84.7 | 88.3 | 44.9 | |
| CoSBackbone=Llama3-70B2026.06 | 0.42 | 86 | 66.8 | 86.7 | 43.9 | |
| LQ (Human)Evaluation=Human counselors in low-quality sessions2026.06 | 0.34 | 16.2 | 14.22 | 40.5 | 59.6 | |
| CoTBackbone=Llama3-70B2026.06 | 0.33 | 90.4 | 28.6 | 78.2 | 48.2 | |
| BaseBackbone=Llama3-70B2026.06 | 0.32 | 82.7 | 12.7 | 57.2 | 47.9 | |
| CoSBackbone=GPT-4o2026.06 | 0.32 | 95.7 | 75.2 | 92.4 | 45.3 | |
| CoTBackbone=GPT-4o2026.06 | 0.3 | 94.7 | 30.8 | 87.2 | 47.2 | |
| BaseBackbone=GPT-4o2026.06 | 0.28 | 91.7 | 18.9 | 84.3 | 42.7 | |
| R1-DistBackbone=Llama3-70B2026.06 | 0.27 | 97.5 | 19.4 | 60.4 | 62.7 |