Disagreement Detection on CEFR-SP Qwen3 labels Rater A
0.593ROC-AUCmost2
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| most2Backbone=ModernBERT [6], Protocol=chain (A)2026.05 | 0.593 | 0.591 | 90 | |
| logprobInput=label, Rater=A2026.05 | 0.567 | 0.539 | 55 | |
| multilingual-e5-large-instruct2026.05 | 0.567 | 0.568 | 95 | |
| deberta-v3-large2026.05 | 0.565 | 0.551 | 80 | |
| LaBSE2026.05 | 0.545 | 0.544 | 95 | |
| all-roberta-large-v12026.05 | 0.544 | 0.536 | 75 | |
| sentence-t5-base2026.05 | 0.543 | 0.537 | 60 | |
| instructor-large2026.05 | 0.542 | 0.539 | 85 | |
| bge-large-en-v1.52026.05 | 0.54 | 0.538 | 75 | |
| gte-large-en-v1.52026.05 | 0.54 | 0.538 | 80 | |
| sentence-t5-large2026.05 | 0.538 | 0.532 | 55 | |
| e5-large-v22026.05 | 0.534 | 0.537 | 80 | |
| exactwVariant=p i, Protocol=chain (A)2026.05 | 0.525 | 0.508 | 60 | |
| exactwBackbone=ModernBERT [6], Protocol=chain (A)2026.05 | 0.482 | 0.509 | 75 |