Dialogue-level classification on CRADLE-DIALOGUE Confirm-only class (test)
79.4Jaccard IndexClaude-4.5-Sonnet
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Claude-4.5-SonnetModel type=Proprietary2026.06 | 79.4 | 75.1 | 66.18 | 83.33 | 75.22 | |
| Qwen3-32b-FTFine-tuning source=Dialogue2026.06 | 79.28 | 76.03 | 72 | 79.96 | 75.62 | |
| Gemini-3-FlashModel type=Proprietary2026.06 | 77.63 | 73.86 | 67.76 | 78.69 | 74.78 | |
| GPT-5.1Model type=Proprietary2026.06 | 76.75 | 73.27 | 66.52 | 77.22 | 72.99 | |
| Qwen2.5-72b-FTFine-tuning source=Reddit Post2026.06 | 75.83 | 71.51 | 58.6 | 77.85 | 66.91 | |
| Llama3.3-70b-FTFine-tuning source=Reddit Post2026.06 | 74.66 | 71.06 | 60.45 | 78.48 | 69.41 | |
| gpt-oss-120bModel type=Instruction-tuned2026.06 | 74.56 | 71.66 | 63.44 | 74.68 | 65.7 | |
| Qwen3-32BModel type=Instruction-tuned2026.06 | 73.49 | 70.56 | 63.75 | 70.04 | 66.9 | |
| Qwen2.5-72B-InstModel type=Instruction-tuned2026.06 | 72.11 | 66.55 | 55.61 | 80.17 | 70.91 | |
| Qwen3-14BModel type=Instruction-tuned2026.06 | 70.65 | 68.08 | 58.48 | 64.35 | 59.22 | |
| Llama-3.3-70B-InstModel type=Instruction-tuned2026.06 | 70.27 | 65.28 | 54.71 | 79.75 | 72.85 | |
| Llama-4-Scout-17BModel type=Instruction-tuned2026.06 | 68.9 | 66.39 | 57.92 | 84.6 | 74.2 | |
| gpt-oss-20bModel type=Instruction-tuned2026.06 | 67.83 | 64.68 | 53.39 | 66.24 | 57.07 | |
| gemma-3-27b-itModel type=Instruction-tuned2026.06 | 66 | 61.28 | 52.6 | 71.94 | 64.51 | |
| gemma-3-12b-itModel type=Instruction-tuned2026.06 | 64.37 | 60 | 49 | 72.78 | 65.43 |