Dialogue-Level Crisis Detection on CRADLE-DIALOGUE
69.09Jaccard IndexGemini-3-Flash
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini-3-FlashCategory=Closed-source Model2026.06 | 69.09 | 68.57 | 65.92 | 69.89 | 71.37 | |
| Claude-4.5-SonnetCategory=Closed-source Model2026.06 | 68.9 | 70.11 | 65.04 | 77.13 | 73.52 | |
| Qwen3-32b-FTFine-tuned data=Dialogue data2026.06 | 66.27 | 68.88 | 69.29 | 73.72 | 72.89 | |
| Qwen3-32BCategory=Open-source Model, Family=Qwen Family2026.06 | 63.98 | 63.43 | 60.89 | 60.37 | 62.82 | |
| gpt-oss-120bCategory=Open-source Model2026.06 | 63.59 | 63.22 | 59.89 | 60.8 | 60.88 | |
| Qwen3-14BCategory=Open-source Model, Family=Qwen Family2026.06 | 62.38 | 59.6 | 54.22 | 52.7 | 54.35 | |
| GPT-5.1Category=Closed-source Model2026.06 | 62.2 | 67 | 65.09 | 75.14 | 72.13 | |
| llama3.3-70b-FTFine-tuned data=Reddit post-level crisis annotations2026.06 | 59.93 | 62.91 | 56.49 | 68.18 | 64.9 | |
| Qwen2.5-72b-FTFine-tuned data=Reddit post-level crisis annotations2026.06 | 59.85 | 64.65 | 56.97 | 75.71 | 65.86 | |
| gpt-oss-20bCategory=Open-source Model2026.06 | 58.38 | 56.39 | 49.42 | 51.42 | 51.86 | |
| Llama-3.3-70B-InstructCategory=Open-source Model, Family=LLaMA Family2026.06 | 56.56 | 59.69 | 51.86 | 70.03 | 68.3 | |
| Qwen2.5-72B-InstructCategory=Open-source Model, Family=Qwen Family2026.06 | 56.08 | 61.36 | 54.16 | 78.27 | 69.43 | |
| gemma-3-27b-itCategory=Open-source Model, Family=Gemma Family2026.06 | 52.03 | 54.45 | 49.22 | 59.94 | 59.66 | |
| Llama-4-Scout-17BCategory=Open-source Model, Family=LLaMA Family2026.06 | 51.98 | 60 | 56.19 | 80.11 | 72.51 | |
| gemma-3-12b-itCategory=Open-source Model, Family=Gemma Family2026.06 | 47.12 | 53.8 | 45.83 | 67.9 | 62.23 |