Natural Language Understanding on GLUE (MNLI, QNLI, SST-2, CoLA Subset)
51.6MNLI ScoreMeZO-Adam
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MeZO-AdamBackbone=RoBERTa-large (355M), Precision=FP322025.11 | 51.6 | 80.1 | 84.8 | 77.9 | 73.6 | |
| LORENBackbone=RoBERTa-large (355M), Precision=FP322025.11 | 44.3 | 76.3 | 86.1 | 73.8 | 70.1 | |
| HiZOOBackbone=RoBERTa-large (355M), Precision=FP322025.11 | 43.1 | 70.2 | 73.2 | 70.2 | 64.2 | |
| MeZO-SVRGBackbone=DistilBERT (66M), Precision=FP322025.11 | 42 | 64 | 73.6 | 66.6 | 61.6 | |
| LOZOBackbone=RoBERTa-large (355M), Precision=FP322025.11 | 41.3 | 70.8 | 54.6 | 69.5 | 59.1 | |
| MeZO-AdamBackbone=DistilBERT (66M), Precision=FP322025.11 | 41.2 | 71.1 | 78.4 | 67.8 | 64.6 | |
| LOZOBackbone=DistilBERT (66M), Precision=FP322025.11 | 40.2 | 53 | 61 | 67 | 55.3 | |
| HiZOOBackbone=DistilBERT (66M), Precision=FP322025.11 | 40 | 64.5 | 78.7 | 67 | 62.6 | |
| MeZOBackbone=DistilBERT (66M), Precision=FP322025.11 | 39.9 | 48.5 | 62.1 | 67 | 54.4 | |
| LORENBackbone=DistilBERT (66M), Precision=FP322025.11 | 39.8 | 73 | 81.7 | 67.2 | 65.4 | |
| MeZOBackbone=RoBERTa-large (355M), Precision=FP322025.11 | 39.8 | 71.6 | 54.8 | 67.2 | 58.4 | |
| MeZO-SVRGBackbone=RoBERTa-large (355M), Precision=FP322025.11 | 39.8 | 59.1 | 55.2 | 67.3 | 54.6 |