Disagreement Detection on CEFR-SP GPT-OSS-120B (Rater B)
55.8ROC-AUClogprob label (B)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| logprob label (B)Detection Strategy=logprob2026.05 | 55.8 | 66.8 | 95 | |
| e5-large-v2Embedding Model=e5-large-v2, Detection Strategy=most22026.05 | 55.2 | 67.6 | 100 | |
| LaBSEEmbedding Model=LaBSE, Detection Strategy=most22026.05 | 55.1 | 68.5 | 100 | |
| multilingual-e5-large-instructEmbedding Model=multilingual-e5-large-instruct, Detection Strategy=most22026.05 | 54.8 | 68.3 | 100 | |
| modernbert most2 / chain (B)Embedding Model=ModernBERT, Detection Strategy=most2, Prompting Style=chain-of-thought2026.05 | 54.7 | 69 | 100 | |
| deberta-v3-largeEmbedding Model=deberta-v3-large, Detection Strategy=most22026.05 | 54 | 67.5 | 100 | |
| all-roberta-large-v1Embedding Model=all-roberta-large-v1, Detection Strategy=most22026.05 | 53.9 | 66.1 | 100 | |
| bge-large-en-v1.5Embedding Model=bge-large-en-v1.5, Detection Strategy=most22026.05 | 53.5 | 66.6 | 100 | |
| gte-large-en-v1.5Embedding Model=gte-large-en-v1.5, Detection Strategy=most22026.05 | 53.5 | 66.5 | 95 | |
| sentence-t5-baseEmbedding Model=sentence-t5-base, Detection Strategy=most22026.05 | 53.1 | 66.5 | 100 | |
| sentence-t5-largeEmbedding Model=sentence-t5-large, Detection Strategy=most22026.05 | 53 | 66.5 | 100 | |
| instructor-largeEmbedding Model=instructor-large, Detection Strategy=most22026.05 | 53 | 66.8 | 100 | |
| p i exactw / chain (B)Detection Strategy=exactw, Prompting Style=chain-of-thought2026.05 | 51.1 | 65.3 | 85 | |
| modernbert exactw / chain (B)Embedding Model=ModernBERT, Detection Strategy=exactw, Prompting Style=chain-of-thought2026.05 | 42.7 | 57.8 | 70 |