Perspectivist Classification on VOICED annotator-level, binary (test)
76.46AccuracyDiADEM
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DiADEMevaluation_mode=affiliation-matched, classification_type=binary2026.04 | 76.46 | 55.87 | 70.25 | 18.26 | 26.16 | 0.0923 | 0.4125 | 22.92 | 1.29 | |
| DisCoevaluation_mode=affiliation-matched, classification_type=binary2026.04 | 73.85 | 42.48 | 62.74 | 0 | 0 | 0.1765 | 0.4989 | 24.95 | 26.15 | |
| Llama-4evaluation_mode=affiliation-matched, classification_type=binary, baseline_type=LLM-as-a-judge2026.04 | 55.17 | 51.94 | 57.99 | 8.87 | 9.99 | 0.3168 | 0.9054 | 45.27 | 44.83 | |
| GPT-5evaluation_mode=affiliation-matched, classification_type=binary, baseline_type=LLM-as-a-judge2026.04 | 54.67 | 51.86 | 57.43 | 9.16 | 10.41 | 0.3319 | 0.9324 | 46.62 | 45.33 | |
| GPT-4o-minievaluation_mode=affiliation-matched, classification_type=binary, baseline_type=LLM-as-a-judge2026.04 | 53.94 | 50.41 | 56.82 | 5.63 | 6.29 | 0.3218 | 0.921 | 46.05 | 46.06 | |
| Gemma-2evaluation_mode=affiliation-matched, classification_type=binary, baseline_type=LLM-as-a-judge2026.04 | 51.14 | 49.16 | 53.99 | 6.18 | 7.37 | 0.324 | 1.0089 | 50.44 | 48.86 | |
| LeWiDievaluation_mode=affiliation-matched, classification_type=binary2026.04 | 26.15 | 20.73 | 10.84 | 0 | 0 | 0.656 | 1.5011 | 75.05 | 73.85 |