Turn-Level Crisis Detection on CRADLE-DIALOGUE
88.62EMClaude-4.5-Sonnet
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Claude-4.5-SonnetCategory=Closed-source Model2026.06 | 88.62 | 89.09 | 56.85 | 56.33 | 63.29 | 64.45 | |
| Gemini-3-FlashCategory=Closed-source Model2026.06 | 88.51 | 88.68 | 53.05 | 52.75 | 54.99 | 58.07 | |
| Qwen3-14BCategory=Open-source Model, Family=Qwen Family2026.06 | 87.48 | 87.77 | 42.53 | 42.27 | 37.83 | 42.8 | |
| gpt-oss-120bCategory=Open-source Model2026.06 | 87.34 | 87.44 | 47.2 | 44.33 | 45.57 | 45.07 | |
| Qwen3-32b-FTFine-tuned data=Dialogue data2026.06 | 86.83 | 87.11 | 51.31 | 54.96 | 54.99 | 57.74 | |
| Qwen3-32BCategory=Open-source Model, Family=Qwen Family2026.06 | 86.33 | 86.57 | 43.75 | 44.28 | 42.05 | 45.79 | |
| gpt-oss-20bCategory=Open-source Model2026.06 | 86.02 | 86.18 | 40.24 | 36.76 | 37.13 | 38.09 | |
| GPT-5.1Category=Closed-source Model2026.06 | 85 | 85.3 | 48.91 | 52.65 | 56.82 | 58.46 | |
| llama3.3-70b-FTFine-tuned data=Reddit post-level crisis annotations2026.06 | 84.56 | 85.05 | 45.64 | 44 | 53.02 | 53.85 | |
| Llama-3.3-70B-InstructCategory=Open-source Model, Family=LLaMA Family2026.06 | 82.57 | 83.55 | 41.74 | 38.81 | 54.57 | 56.64 | |
| gemma-3-27b-itCategory=Open-source Model, Family=Gemma Family2026.06 | 82.35 | 82.9 | 37.1 | 36.17 | 45.99 | 47.94 | |
| Qwen2.5-72b-FTFine-tuned data=Reddit post-level crisis annotations2026.06 | 78.42 | 79.5 | 40.32 | 38.66 | 59.35 | 53.92 | |
| gemma-3-12b-itCategory=Open-source Model, Family=Gemma Family2026.06 | 73.62 | 74.53 | 30.65 | 29.26 | 48.1 | 46.89 | |
| Llama-4-Scout-17BCategory=Open-source Model, Family=LLaMA Family2026.06 | 73.34 | 75.06 | 33.25 | 34.08 | 59.35 | 57.13 | |
| Qwen2.5-72B-InstructCategory=Open-source Model, Family=Qwen Family2026.06 | 70.18 | 71.95 | 33.03 | 33.15 | 64.56 | 59.17 |