Natural Language Understanding on GLUE (All Tasks)
95.18SST-2SIFT
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SIFTPre-trained Backbone=RoBERTa-Base, Evaluation Protocol=Fine-tuning2026.03 | 95.18 | 62.39 | 92.73 | 77.98 | 90.28 | 92.59 | 88.72 | — | — | 85.91 | 87.4 | — | — | — | — | — | — | |
| RoBERTaPrecision (W/A)=32-322025.11 | 95.18 | — | 90.44 | 80.51 | — | 92.68 | 91.6 | 87.75 | 87.23 | 86.4 | — | — | 62.5 | 93.1 | 88.78 | 91.04 | 90.72 | |
| AdamWPre-trained Backbone=RoBERTa-Base, Evaluation Protocol=Fine-tuning, Parameters=Full params2026.03 | 94.84 | 64.16 | 92.07 | 80.51 | 90.81 | 92.93 | 89.12 | — | — | 86.55 | 87.97 | — | — | — | — | — | — | |
| LISA-wor-no-scalePre-trained Backbone=RoBERTa-Base, Evaluation Protocol=Fine-tuning, Sampling=Without Replacement, Gradient Scaling=False2026.03 | 94.84 | 62.35 | 92.36 | 78.34 | 90.45 | 92.59 | 88.73 | — | — | 85.9 | 87.55 | — | — | — | — | — | — | |
| GoLorePre-trained Backbone=RoBERTa-Base, Evaluation Protocol=Fine-tuning2026.03 | 94.72 | 62.62 | 91.95 | 78.7 | 90.49 | 92.35 | 87.83 | — | — | 85.75 | 87.33 | — | — | — | — | — | — | |
| LISA-worPre-trained Backbone=RoBERTa-Base, Evaluation Protocol=Fine-tuning, Sampling=Without Replacement, Gradient Scaling=True2026.03 | 94.72 | 62.98 | 92.82 | 79.06 | 90.49 | 92.88 | 88.73 | — | — | 86.18 | 87.72 | — | — | — | — | — | — | |
| LISAPre-trained Backbone=RoBERTa-Base, Evaluation Protocol=Fine-tuning2026.03 | 94.5 | 61.76 | 92.25 | 78.34 | 90.19 | 92.68 | 88.77 | — | — | 85.75 | 87.54 | — | — | — | — | — | — | |
| LISA-scalePre-trained Backbone=RoBERTa-Base, Evaluation Protocol=Fine-tuning, Gradient Scaling=True2026.03 | 94.27 | 61.51 | 91.91 | 76.17 | 90.2 | 92.71 | 88.81 | — | — | 85.39 | 87.55 | — | — | — | — | — | — | |
| MASS2025.04 | 94.2 | 74.1 | 85.8 | 84.9 | 88.9 | 90.9 | 85.1 | — | — | 85.9 | 83.2 | 99.4 | — | — | — | — | — | |
| Finetuned2025.04 | 93.6 | 75 | 87.5 | 85.9 | 88.7 | 91.5 | 85.4 | — | — | 86.4 | 83.4 | 100 | — | — | — | — | — | |
| SMILE-22025.04 | 93.5 | 73.2 | 85 | 84.8 | 87.3 | 91.3 | 84.9 | — | — | 85.5 | 84.2 | 99 | — | — | — | — | — | |
| BERTPrecision (W/A)=32-322025.11 | 93.35 | — | 87.75 | 72.56 | — | 91.84 | 90.91 | 84.94 | 84.76 | 83.83 | — | — | 59.6 | 91.35 | 87.82 | 89.7 | 89.22 | |
| SMILE-12025.04 | 93.3 | 72 | 84.3 | 84.1 | 87 | 91.3 | 84.7 | — | — | 85.1 | 84.2 | 98.5 | — | — | — | — | — | |
| QUARKBackbone=RoBERTa, Precision (W/A)=6-62025.11 | 92.72 | — | 83.42 | 78.75 | — | 86.98 | 89.03 | 83.33 | 82.68 | 81.77 | — | — | 49.49 | 87.65 | 86.34 | 90.02 | 89.92 | |
| BERT-BaseTruncation Ratio=0%2025.12 | 92.5 | 56.7 | 89.8 | 65.8 | 85.7 | 90.3 | 87.6 | 82.7 | 83.2 | 81.6 | — | — | — | — | — | — | — | |
| Task Arithmetic2025.04 | 92.3 | 70.5 | 78.4 | 80.5 | 77.8 | 90.2 | 83.6 | — | — | 78.9 | 57.8 | 91.3 | — | — | — | — | — | |
| Outlier Sup.Backbone=RoBERTa, Precision (W/A)=6-62025.11 | 92.2 | — | 83.33 | 67.51 | — | 86.82 | 90.01 | 83.38 | 83.32 | 79.62 | — | — | 46.35 | 87.5 | 86.86 | 86.83 | 86.93 | |
| Outlier Sup.Backbone=BERT, Precision (W/A)=6-62025.11 | 91.86 | — | 83.33 | 70.76 | — | 89.82 | 88.94 | 82.02 | 81.69 | 81.19 | — | — | 54.4 | 87.45 | 84.69 | 88.65 | 88.55 | |
| Weight Averaging2025.04 | 91.7 | 69.1 | 79.4 | 81.2 | 73.2 | 89.8 | 83.9 | — | — | 78.9 | 62.6 | 91.3 | — | — | — | — | — | |
| Ties-Merging2025.04 | 91.7 | 70.3 | 78.9 | 81.6 | 78.3 | 90.2 | 83.5 | — | — | 79.9 | 65 | 92.5 | — | — | — | — | — | |
| bert2BERTModel=RoBERTa-B, FLOPs Savings=7.2%, Walltime Savings=6.6%2026.03 | 91.63 | 41.04 | 86.03 | — | 85.2 | 89 | 90.21 | — | — | 80.73 | 82 | — | — | — | — | — | — | |
| Prehab-SVDTruncation Ratio=20%2025.12 | 91.4 | 49.2 | 90.1 | 61.4 | 85.2 | 87.6 | 85.3 | 81.2 | 81.8 | 79 | — | — | — | — | — | — | — | |
| QUARKBackbone=BERT, Precision (W/A)=6-62025.11 | 91.09 | — | 83.56 | 70.15 | — | 90.2 | 89.83 | 82.78 | 81.61 | 82.25 | — | — | 61.41 | 88.19 | 86.88 | 88.38 | 87.5 | |
| MangoModel=RoBERTa-B, FLOPs Savings=22.2%, Walltime Savings=20.4%2026.03 | 90.83 | 44.36 | 84.31 | — | 85.87 | 90.43 | 90.23 | — | — | 81.15 | 81.99 | — | — | — | — | — | — | |
| ScratchModel=RoBERTa-B, FLOPs Savings=0.0%, Walltime Savings=0.0%2026.03 | 90.71 | 45.87 | 83.33 | — | 85.44 | 89.75 | 90.38 | — | — | 81.15 | 82.59 | — | — | — | — | — | — | |
| BoTModel=RoBERTa-B, FLOPs Savings=54.7%, Walltime Savings=51.5%2026.03 | 90.6 | 44 | 84.82 | — | 85.7 | 90.13 | 90.19 | — | — | 81.19 | 82.91 | — | — | — | — | — | — | |
| Prehab-SVDTruncation Ratio=40%2025.12 | 90.5 | 42.1 | 88.7 | 58.1 | 82.1 | 81.8 | 83.8 | 79 | 78.9 | 75.8 | — | — | — | — | — | — | — | |
| LiGOModel=RoBERTa-B, FLOPs Savings=22.1%, Walltime Savings=20.3%2026.03 | 90.37 | 44.96 | 83.82 | — | 85.16 | 90.19 | 90.18 | — | — | 81.01 | 82.38 | — | — | — | — | — | — | |
| QUARKBackbone=RoBERTa, Precision (W/A)=4-42025.11 | 90.28 | — | 66.72 | 73.95 | — | 52.45 | 58.39 | 36.26 | 37.33 | 59.02 | — | — | 10.49 | 75.44 | 40.75 | 88.71 | 87.03 | |
| SVD-LLMTruncation Ratio=20%2025.12 | 89.8 | 45.2 | 88.4 | 52.7 | 84.7 | 87.1 | 84.7 | 80.8 | 81.2 | 76.7 | — | — | — | — | — | — | — | |
| ScratchModel=RoBERTa-S, FLOPs Savings=0.0%, Walltime Savings=0.0%2026.03 | 89.56 | 20.39 | 79.41 | — | 82.59 | 84.9 | 88.32 | — | — | 74.54 | 76.64 | — | — | — | — | — | — | |
| EasyQuantBackbone=RoBERTa, Precision (W/A)=6-62025.11 | 89.56 | — | 76.47 | 62.45 | — | 74.04 | 89.12 | 74.96 | 75.87 | 70.01 | — | — | 9.28 | 84.31 | 85.52 | 80.89 | 82.38 | |
| BoTModel=RoBERTa-S, FLOPs Savings=35.7%, Walltime Savings=35.8%2026.03 | 89.33 | 23.18 | 76.47 | — | 82.93 | 84.61 | 88.71 | — | — | 74.57 | 76.78 | — | — | — | — | — | — | |
| SVD-LLMTruncation Ratio=40%2025.12 | 89.1 | 41 | 86.9 | 48.6 | 80.9 | 78.4 | 82.9 | 78.1 | 77.2 | 73.2 | — | — | — | — | — | — | — | |
| FWSVDTruncation Ratio=20%2025.12 | 89 | 28.7 | 82.5 | 55.2 | 82.1 | 86.2 | 83.8 | 70.5 | 69.8 | 72.2 | — | — | — | — | — | — | — | |
| GFWSVDTruncation Ratio=20%2025.12 | 88.8 | 39.9 | 85.5 | 52.3 | 64.4 | 82.8 | 82.8 | 71.8 | 72.3 | 71.1 | — | — | — | — | — | — | — | |
| WeMoE2025.04 | 88.6 | 72.5 | 51.9 | 81.5 | 82.1 | 89.3 | 69.6 | — | — | 76.8 | 79 | 88.9 | — | — | — | — | — | |
| WSModel=RoBERTa-S, FLOPs Savings=32.4%, Walltime Savings=32.3%2026.03 | 88.5 | 21.62 | 78.19 | — | 81.43 | 83.54 | 88.15 | — | — | 74.04 | 76.85 | — | — | — | — | — | — | |
| PercentileBackbone=RoBERTa, Precision (W/A)=6-62025.11 | 88.19 | — | 78.43 | 62.82 | — | 77.16 | 87.44 | 72.23 | 73.68 | 70.98 | — | — | 20.73 | 84.43 | 82.21 | 79.41 | 79.64 | |
| Prehab-SVDTruncation Ratio=60%2025.12 | 87.7 | 30.9 | 84.7 | 56.7 | 68.1 | 67.1 | 77.1 | 69.8 | 69.5 | 67.7 | — | — | — | — | — | — | — | |
| PercentileBackbone=BERT, Precision (W/A)=6-62025.11 | 87.27 | — | 79.9 | 61.73 | — | 79.37 | 80.19 | 72.4 | 71.69 | 72.93 | — | — | 37.32 | 85.09 | 72.58 | 86.38 | 87.29 | |
| EasyQuantBackbone=BERT, Precision (W/A)=6-62025.11 | 87.27 | — | 77.45 | 65.34 | — | 84.94 | 81.81 | 75.82 | 75.66 | 74.49 | — | — | 38.16 | 82.51 | 75.31 | 85.5 | 86.33 | |
| GFWSVDTruncation Ratio=40%2025.12 | 85.2 | 21.9 | 72.4 | 46.2 | 60.1 | 80.9 | 80.9 | 63.2 | 63.3 | 63.8 | — | — | — | — | — | — | — | |
| QUARKBackbone=BERT, Precision (W/A)=4-42025.11 | 84.39 | — | 67.12 | 51.77 | — | 67.08 | 72.76 | 39.15 | 40.64 | 59.91 | — | — | 43.76 | 80.43 | 35.99 | 63.67 | 64.82 | |
| SVDTruncation Ratio=20%2025.12 | 84.1 | 2.1 | 68.4 | 52 | 61.6 | 71.7 | 72.7 | 57.9 | 58.3 | 58.8 | — | — | — | — | — | — | — | |
| SVD-LLMTruncation Ratio=60%2025.12 | 83.6 | 25.9 | 83 | 52 | 66.2 | 65.1 | 76.3 | 65.8 | 65.9 | 64.7 | — | — | — | — | — | — | — | |
| FWSVDTruncation Ratio=40%2025.12 | 82.8 | 0 | 82.3 | 50.2 | 81 | 81.7 | 80.1 | 54.6 | 54.9 | 64.1 | — | — | — | — | — | — | — | |
| FWSVDTruncation Ratio=60%2025.12 | 79.9 | 0 | 81.1 | 51.6 | 68 | 66.7 | 71 | 44.5 | 44.7 | 58.4 | — | — | — | — | — | — | — | |
| SVDTruncation Ratio=40%2025.12 | 78.9 | 0 | 7.5 | 50.9 | 43.7 | 64.5 | 59.6 | 45.1 | 45.6 | 43.8 | — | — | — | — | — | — | — | |
| GFWSVDTruncation Ratio=60%2025.12 | 73.7 | 7.2 | 0.7 | 46.9 | 67.7 | 67 | 68 | 47.2 | 48.5 | 47.4 | — | — | — | — | — | — | — | |
| Outlier Sup.Backbone=BERT, Precision (W/A)=4-42025.11 | 70.41 | — | 60.78 | 52.71 | — | 59.64 | 65.87 | 27.09 | 26.55 | 39.5 | — | — | -1.63 | 68 | 23.34 | 0.48 | -0.29 | |
| SVDTruncation Ratio=60%2025.12 | 62.7 | 0 | 0 | 52.3 | 11.4 | 56.1 | 49.8 | 35.6 | 35.2 | 33.5 | — | — | — | — | — | — | — | |
| Outlier Sup.Backbone=RoBERTa, Precision (W/A)=4-42025.11 | 51.15 | — | 63.84 | 52.35 | — | 49.5 | 60.36 | 32.75 | 32.37 | 35.98 | — | — | -3.24 | 76.76 | 15.75 | -3.15 | -2.52 | |
| PercentileBackbone=BERT, Precision (W/A)=4-42025.11 | 50.92 | — | 68.38 | 47.29 | — | 50.36 | 60.71 | 32.73 | 32.95 | 35.64 | — | — | -4.15 | 81.22 | 23.16 | -9.18 | -8.63 | |
| PercentileBackbone=RoBERTa, Precision (W/A)=4-42025.11 | 50.92 | — | 63.38 | 47.29 | — | 50.54 | 52.92 | 31.82 | 31.72 | 36.89 | — | — | 0.01 | 77.22 | 38.15 | -1.52 | -0.99 |