Natural Language Understanding on GLUE Frozen (val)
76.9MNLICrossBERT
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| CrossBERTModel Scale=1B, Compute=1e21, Training Setup=Standard, Evaluation Protocol=Frozen Glue2026.07 | 76.9 | 89 | 85.4 | 72.5 | 93.4 | 81.6 | 61 | 88.2 | 81 | |
| ElectraModel Scale=1B, Compute=1e21, Training Setup=Different hyperparameters, Evaluation Protocol=Frozen Glue2026.07 | 75.2 | 85.9 | 85.5 | 68.9 | 91.2 | 82.1 | 65.2 | 89.4 | 80.4 | |
| BERTModel Scale=2B, Training Setup=Different hyperparameters, Evaluation Protocol=Frozen Glue2026.07 | 69.9 | 84.9 | 77.6 | 68.6 | 79.6 | 77.9 | 40.1 | 79.1 | 72.2 | |
| BERTModel Scale=1B, Compute=1e21, Training Setup=Standard, Evaluation Protocol=Frozen Glue2026.07 | 60.3 | 78.6 | 73.5 | 60.3 | 71.2 | 70.8 | 19.3 | 70.1 | 63 | |
| BERTModel Scale=1B, Compute=1e21, Training Setup=Different hyperparameters, Evaluation Protocol=Frozen Glue2026.07 | 59.6 | 81.3 | 76.6 | 63.9 | 79.7 | 72.3 | 12.3 | 71.6 | 64.6 | |
| ElectraModel Scale=1B, Compute=1e21, Training Setup=Standard, Evaluation Protocol=Frozen Glue2026.07 | 59 | 80.7 | 78.6 | 64.6 | 72.5 | 80.4 | 16.7 | 79.8 | 66.5 |