Natural Language Understanding on GLUE (SST-2 and MNLI Subset)
97.1SST-2 AccuracyModernBERT large
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ModernBERT largeModel Size Category=Large, Parameters=353-395M2025.07 | 97.1 | 90.8 | 90.4 | |
| DeBERTa-v1-xlModel Size Category=XL, Parameters=884M-1.2B2025.07 | 97.1 | 91.7 | 90.7 | |
| Ettin-Enc-1BModel Size Category=XL, Parameters=884M-1.2B2025.07 | 97.1 | 91.8 | 91.6 | |
| Ettin-Enc-400mModel Size Category=Large, Parameters=353-395M2025.07 | 96.7 | 91.3 | 90.8 | |
| ModernBERT baseModel Size Category=Base, Parameters=123-150M2025.07 | 96 | 89.1 | 88.4 | |
| Ettin-Enc-150mModel Size Category=Base, Parameters=123-150M2025.07 | 95.8 | 89.2 | 88.9 | |
| Ettin-Enc-68mModel Size Category=Small, Parameters=68-82M2025.07 | 94.4 | 87 | 87.2 | |
| MiniLM L12Model Size Category=XS, Parameters=28-33M2025.07 | 93.3 | 85.6 | 84.6 | |
| BERT largeModel Size Category=Large, Parameters=353-395M2025.07 | 93.3 | 86.3 | 85.2 | |
| DistilRoBERTaModel Size Category=Small, Parameters=68-82M2025.07 | 93.1 | 84.7 | 83.8 | |
| BERT baseModel Size Category=Base, Parameters=123-150M2025.07 | 93.1 | 85.4 | 84.7 | |
| DistilBERTModel Size Category=Small, Parameters=68-82M2025.07 | 92.2 | 82.7 | 81.5 | |
| Ettin-Enc-32mModel Size Category=XS, Parameters=28-33M2025.07 | 92 | 83.4 | 83.5 | |
| TinyBERTModel Size Category=XXS, Parameters=7-17M2025.07 | 91.2 | 80.9 | 77 | |
| Ettin-Enc-17mModel Size Category=XXS, Parameters=7-17M2025.07 | 91.2 | 79.5 | 79.2 | |
| BERT-smallModel Size Category=XS, Parameters=28-33M2025.07 | 90.1 | 79.2 | 79 | |
| BERT-miniModel Size Category=XXS, Parameters=7-17M2025.07 | 88.3 | 77.2 | 76.4 |