Natural Language Understanding on GLUE (MNLI, QNLI, QQP, RTE, SST, MRPC, CoLA, STS)
91.9MNLI AccuracyDeBERTaV3
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DeBERTaV3Params=304M, FLOPS=>2e20, Evaluation Protocol=Full-finetuning (unfrozen backbone)2026.07 | 91.9 | 96 | 93 | 92.7 | 96.9 | 91.9 | 75.3 | 93 | 91.4 | |
| ModernBERTParams=352M, FLOPS=4.9e21, Evaluation Protocol=Full-finetuning (unfrozen backbone)2026.07 | 90.8 | 95.2 | 92.7 | 92.1 | 97.1 | 91.7 | 71.4 | 92.8 | 90.5 | |
| NeoBERTParams=198M, FLOPS=2.9e21, Evaluation Protocol=Full-finetuning (unfrozen backbone)2026.07 | 88.9 | 93.9 | 90.7 | 91 | 95.8 | 93.4 | 64.8 | 92.1 | 88.8 | |
| ElectraParams=258M, FLOPS=7.5e19, tps=116k, Evaluation Protocol=Full-finetuning (unfrozen backbone)2026.07 | 88.7 | 93.3 | 89.1 | 83.4 | 94.6 | 92 | 71 | 90 | 87.8 | |
| OptiBERTParams=239M, FLOPS=7.0e19, Evaluation Protocol=Full-finetuning (unfrozen backbone)2026.07 | 86.6 | 92.1 | 90.3 | 83.2 | 92.6 | 91 | 59.6 | 90.8 | 85.8 | |
| BERTParams=239M, FLOPS=6.8e19, tps=123k, Evaluation Protocol=Full-finetuning (unfrozen backbone)2026.07 | 86.5 | 90 | 88.5 | 85.3 | 95.1 | 91.9 | 63 | 91.2 | 86.4 | |
| CrossBERTParams=279M, FLOPS=4.1e19, tps=207k, Evaluation Protocol=Full-finetuning (unfrozen backbone)2026.07 | 86.4 | 90.3 | 88.4 | 85.1 | 94.8 | 91.4 | 65.5 | 91.5 | 86.6 | |
| DeBERTaV3Params=304M, FLOPS=>2e20, Evaluation Protocol=Linear probe2026.07 | 76.6 | 86.9 | 85.8 | 75.8 | 86.5 | 81.9 | 67.1 | 90.1 | 81.3 | |
| ElectraParams=258M, FLOPS=7.5e19, tps=116k, Evaluation Protocol=Linear probe2026.07 | 69.2 | 83.6 | 82.7 | 66.8 | 86.9 | 79.6 | 58.8 | 87.5 | 76.9 | |
| ModernBERTParams=352M, FLOPS=4.9e21, Evaluation Protocol=Linear probe2026.07 | 63.7 | 80 | 80.7 | 61 | 86.7 | 74 | 40.7 | 83 | 71.2 | |
| CrossBERTParams=279M, FLOPS=4.1e19, tps=207k, Evaluation Protocol=Linear probe2026.07 | 61.6 | 81.8 | 81.4 | 64.3 | 90.6 | 77.9 | 47.9 | 84.8 | 73.8 | |
| BERTParams=239M, FLOPS=6.8e19, tps=123k, Evaluation Protocol=Linear probe2026.07 | 58.6 | 74.8 | 77.7 | 59.9 | 84.9 | 76.7 | 27.4 | 79.5 | 67.4 | |
| NeoBERTParams=198M, FLOPS=2.9e21, Evaluation Protocol=Linear probe2026.07 | 47.9 | 69.8 | 72.9 | 56 | 78 | 69.1 | 10.1 | 66.1 | 58.7 |