Natural Language Understanding on GLUE (CoLA, MNLI, MRPC, QNLI, QQP, RTE, SST2, STSB)
62.33CoLA ScoreFull
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| FullPre-trained Model=RoBERTa-Base, Optimizer=AdamW, Evaluation Protocol=Full finetuning2025.06 | 62.33 | 87.62 | 91.11 | 92.92 | 90.26 | 75.81 | 95.18 | 90.5 | 85.72 | |
| MLorcPre-trained Model=RoBERTa-Base, Rank=8, Optimizer=AdamW, Evaluation Protocol=Memory-efficient fine-tuning2025.06 | 62.07 | 87.53 | 90.77 | 93.19 | 88.99 | 77.98 | 95.18 | 90.59 | 85.79 | |
| LoRAPre-trained Model=RoBERTa-Base, Rank=8, Optimizer=AdamW, Evaluation Protocol=Memory-efficient fine-tuning2025.06 | 61.53 | 87.51 | 90.1 | 92.75 | 89.45 | 76.53 | 94.72 | 90.74 | 85.42 | |
| LDAdamWPre-trained Model=RoBERTa-Base, Rank=8, Optimizer=LDAdamW, Evaluation Protocol=Memory-efficient fine-tuning2025.06 | 60.82 | 87.33 | 90.35 | 93.03 | 90.06 | 76.53 | 94.61 | 90.74 | 85.43 | |
| GaLorePre-trained Model=RoBERTa-Base, Rank=8, Optimizer=AdamW, Evaluation Protocol=Memory-efficient fine-tuning2025.06 | 60.34 | 86.84 | 90.1 | 92.42 | 88.15 | 71.12 | 94.38 | 90.5 | 84.23 |