Natural Language Understanding on GLUE (7-Task Subset: CoLA, SST-2, MRPC, STS-B, QQP, MNLI, QNLI)
88.15Average ScoreModernBERT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ModernBERTWindow Size Multiplier=×22025.11 | 88.15 | 78.98 | 93.08 | 87.93 | 89.11 | 88.26 | 87.21 | 92.48 | 22.6 | |
| RoformerWindow Size Multiplier=×22025.11 | 87.72 | 84.53 | 91.83 | 88.47 | 87.96 | 87.61 | 83.58 | 90.03 | 40.4 | |
| DeepCoT RoformerWindow Size Multiplier=×22025.11 | 85.18 | 81.35 | 89.33 | 84.31 | 87.54 | 85.93 | 80.93 | 86.88 | 95.3 | |
| DeepCoT RoformerWindow Size Multiplier=×12025.11 | 84.8 | 80.91 | 89.23 | 85.95 | 86.48 | 85.79 | 79.04 | 86.18 | 84.4 | |
| RoformerWindow Size Multiplier=×12025.11 | 83.39 | 84.64 | 89.59 | 85.01 | 83.22 | 85.93 | 75.74 | 79.61 | 53.6 | |
| ModernBERTWindow Size Multiplier=×12025.11 | 83.13 | 77.21 | 89.39 | 84.69 | 82.28 | 87.17 | 80.23 | 80.94 | 32.7 | |
| FNetWindow Size Multiplier=×22025.11 | 82.6 | 80.54 | 89.11 | 83.14 | 84.31 | 84.03 | 73.87 | 83.2 | 45.8 | |
| DeepCoT RoformerWindow Size Multiplier=×0.52025.11 | 80.81 | 80.97 | 87.93 | 80.9 | 79.84 | 82.8 | 74.76 | 78.44 | 86.4 | |
| FNetWindow Size Multiplier=×12025.11 | 77.79 | 81.64 | 84.68 | 77.11 | 81.67 | 81.42 | 65.58 | 72.41 | 70.1 | |
| DeepCoT SOFT RoformerWindow Size Multiplier=×22025.11 | 77.68 | 81.04 | 82.98 | 82.19 | 74.12 | 80.33 | 72.21 | 70.88 | 66.3 | |
| DeepCoT SOFT RoformerWindow Size Multiplier=×12025.11 | 76.83 | 81.82 | 82.7 | 81.19 | 73.52 | 79.46 | 70.33 | 68.76 | 74.9 | |
| SOFT RoformerWindow Size Multiplier=×22025.11 | 76.63 | 80.4 | 83.08 | 81.89 | 73.75 | 80.56 | 71.02 | 65.69 | 28.5 | |
| DeepCoT SOFT RoformerWindow Size Multiplier=×0.52025.11 | 74.71 | 81.4 | 83.67 | 80 | 72.27 | 76.08 | 66.4 | 63.14 | 73.7 | |
| ModernBERTWindow Size Multiplier=×0.52025.11 | 72.66 | 75.79 | 81.48 | 79.8 | 77.27 | 77.2 | 57.47 | 59.58 | 49.4 | |
| SOFT RoformerWindow Size Multiplier=×12025.11 | 72.59 | 79.13 | 79.91 | 76.66 | 72.79 | 78.28 | 63.53 | 57.8 | 39.3 | |
| RoformerWindow Size Multiplier=×0.52025.11 | 69.52 | 79.58 | 79.42 | 80.51 | 76.57 | 76.07 | 35.33 | 59.18 | 80.1 | |
| SOFT RoformerWindow Size Multiplier=×0.52025.11 | 69.01 | 79.42 | 72.02 | 80.53 | 69.82 | 72.35 | 56.59 | 52.34 | 57 | |
| FNetWindow Size Multiplier=×0.52025.11 | 67.71 | 81.75 | 77.86 | 76.35 | 74.47 | 74.17 | 31.82 | 57.54 | 112.3 |