Perspectivist Classification on DICES 3-class annotator-level
73.37AccuracyDiADEM
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DiADEM2026.04 | 73.37 | 41.16 | 69.8 | 0.245 | 0.2643 | 0.0446 | 0.7795 | 49.11 | 0.0391 | |
| Llama-4type=LLM baseline2026.04 | 66.8 | 37.58 | 63.98 | 0.118 | 0.1252 | 0.3313 | 0.6626 | 59.9 | 0.332 | |
| GPT-4o-minitype=LLM baseline2026.04 | 66.13 | 37.06 | 64.57 | 0.1391 | 0.1419 | 0.3381 | 0.6761 | 61.94 | 0.3387 | |
| GPT-5type=LLM baseline2026.04 | 66.09 | 35.78 | 63.34 | 0.0927 | 0.096 | 0.3384 | 0.6768 | 64.95 | 0.3391 | |
| LeWiDi2026.04 | 65.62 | 26.41 | 52 | 0 | 0 | 0.0254 | 0.8954 | 63.23 | 0.0377 | |
| DisCo2026.04 | 62.92 | 32.61 | 60.02 | 0.003 | 0.003 | 0.0809 | 0.901 | 70.36 | 0.071 | |
| Gemma-2type=LLM baseline2026.04 | 53.58 | 37.37 | 59.35 | 0.1464 | 0.1577 | 0.4636 | 0.9273 | 70.3 | 0.4642 |