Natural Language Understanding on GLUE (Full Evaluation)
11.01CoLA ScoreEmbBERT
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EmbBERT2025.02 | 11.01 | 79.33 | 69.19 | 83.25 | 67.83 | 68.63 | 68.92 | 49.96 | 87.61 | 49.25 | 63.5 | |
| Embedder2025.02 | 9.65 | 78.9 | 62.25 | 83.28 | 62.06 | 62.17 | 65.4 | 52.73 | 77.2 | 15.58 | 56.92 | |
| Embedder + conv2025.02 | 9.25 | 79.1 | 60.5 | 82.98 | 61.98 | 60.93 | 62.08 | 52 | 79.16 | 16.1 | 56.41 | |
| NanoBERTSize=2MB2025.02 | 9.04 | 78.82 | 65.04 | 79.96 | 63.08 | 63.3 | 63.2 | 51.76 | 87.3 | 13.46 | 57.5 | |
| MAMBASize=2MB2025.02 | 2.56 | 81.16 | 64.62 | 79.18 | 61.22 | 61.4 | 63.2 | 50.2 | 23.38 | 10.16 | 49.71 | |
| BERT-TinySize=20MB2025.02 | 0 | 83.2 | 71.1 | 62.2 | 70.2 | 70.3 | 81.5 | 5.72 | 62.3 | 73.6 | 63.16 | |
| BERTSize=2MB2025.02 | -0.86 | 71.28 | 64.66 | 73.04 | 60.56 | 61.58 | 60.82 | 48.24 | 66.2 | 15.48 | 52.1 |