Validating Response Identification on M-EDESConv automatic annotation
85.28Overall M-F1XLM-RoBERTa
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| XLM-RoBERTa2026.06 | 85.28 | 80.66 | 93.64 | 86.67 | |
| mBERT2026.06 | 74.3 | 70.24 | 76.32 | 73.15 | |
| GPT 4.1 NanoEvaluation Protocol=3-shot2026.06 | 66.57 | 50.36 | 74.6 | 60.13 | |
| GPT 4.1 NanoEvaluation Protocol=Zero-shot2026.06 | 57.98 | 42.71 | 85.19 | 56.89 | |
| Random Baseline2026.06 | 48.38 | 32.45 | 50.13 | 39.4 | |
| Llama 3.1 8bEvaluation Protocol=Zero-shot2026.06 | 41.04 | 34.37 | 85.61 | 49.04 | |
| Llama 3.1 8bEvaluation Protocol=LoRA2026.06 | 32.96 | 37.16 | 97.02 | 53.74 | |
| Llama 3.1 8bEvaluation Protocol=3-shot2026.06 | 32.18 | 33.82 | 97.88 | 50.27 |