Text Classification on GLUE
87.6Average ScorePACE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| PACEBackbone=RoBERTa-base2024.09 | 87.6 | 66.2 | 92 | 91.4 | 86.9 | 93.6 | 95.6 | |
| LoRABackbone=RoBERTa-base2024.09 | 86.6 | 63.4 | 91.5 | 89.7 | 86.6 | 93.3 | 95.1 | |
| BitFitBackbone=RoBERTa-base2024.09 | 85.4 | 62 | 90.8 | 92.7 | 81.5 | 91.8 | 93.7 | |
| FullBackbone=RoBERTa-base2024.09 | 85.2 | 63.6 | 91.2 | 90.2 | 78.7 | 92.8 | 94.8 | |
| VeRABackbone=RoBERTa-base2024.09 | 85.2 | 65.6 | 90.7 | 89.5 | 78.7 | 91.8 | 94.6 | |
| IndividualBackbone=RoBERTa2025.03 | 84.83 | — | — | — | — | — | — | |
| AdaptBackbone=RoBERTa-base2024.09 | 84.2 | 62.6 | 90.3 | 88.4 | 75.9 | 93 | 94.7 | |
| IndividualBackbone=GPT-22025.03 | 76.8 | — | — | — | — | — | — | |
| CART+AdaRankBackbone=RoBERTa2025.03 | 75.47 | — | — | — | — | — | — | |
| TSV-M+AdaRankBackbone=RoBERTa2025.03 | 73.09 | — | — | — | — | — | — | |
| TSV-M+AdaMergingBackbone=RoBERTa2025.03 | 70.65 | — | — | — | — | — | — | |
| TA+AdaRankBackbone=RoBERTa2025.03 | 70.32 | — | — | — | — | — | — | |
| CARTBackbone=RoBERTa2025.03 | 69.97 | — | — | — | — | — | — | |
| CART+AdaMergingBackbone=RoBERTa2025.03 | 69.54 | — | — | — | — | — | — | |
| TA+AdaMergingBackbone=RoBERTa2025.03 | 67.62 | — | — | — | — | — | — | |
| TSV-M+AdaRankBackbone=GPT-22025.03 | 67.43 | — | — | — | — | — | — | |
| Task Arithmetic (TA)Backbone=RoBERTa2025.03 | 67.18 | — | — | — | — | — | — | |
| TSV-MBackbone=RoBERTa2025.03 | 66.93 | — | — | — | — | — | — | |
| CART+AdaRankBackbone=GPT-22025.03 | 65.87 | — | — | — | — | — | — | |
| Ties-MergingBackbone=RoBERTa2025.03 | 64.44 | — | — | — | — | — | — | |
| TA+AdaRankBackbone=GPT-22025.03 | 63.28 | — | — | — | — | — | — | |
| TSV-M+AdaMergingBackbone=GPT-22025.03 | 62.19 | — | — | — | — | — | — | |
| CART+AdaMergingBackbone=GPT-22025.03 | 62.07 | — | — | — | — | — | — | |
| TSV-MBackbone=GPT-22025.03 | 61.95 | — | — | — | — | — | — | |
| CARTBackbone=GPT-22025.03 | 61.82 | — | — | — | — | — | — | |
| Ties-MergingBackbone=GPT-22025.03 | 61.12 | — | — | — | — | — | — | |
| Task Arithmetic (TA)Backbone=GPT-22025.03 | 60.64 | — | — | — | — | — | — | |
| TA+AdaMergingBackbone=GPT-22025.03 | 59.97 | — | — | — | — | — | — | |
| CurvZOModel=OPT-6.7B, Number of training samples=1,000, Optimization Method=CurvZO, Fine-tuning Protocol=Full fine-tuning2026.03 | — | — | — | — | — | — | 94.7 | |
| CurvZO + LoRAModel=OPT-6.7B, Number of training samples=1,000, Optimization Method=CurvZO, Fine-tuning Protocol=LoRA2026.03 | — | — | — | — | — | — | 93.1 | |
| DiZOModel=OPT-6.7B, Number of training samples=1,000, Optimization Method=DiZO, Fine-tuning Protocol=Full fine-tuning2026.03 | — | — | — | — | — | — | 91.6 | |
| DiZO + LoRAModel=OPT-6.7B, Number of training samples=1,000, Optimization Method=DiZO, Fine-tuning Protocol=LoRA2026.03 | — | — | — | — | — | — | 91.4 | |
| MeZOModel=OPT-6.7B, Number of training samples=1,000, Optimization Method=MeZO, Fine-tuning Protocol=Full fine-tuning2026.03 | — | — | — | — | — | — | 94.1 | |
| MeZO + LoRAModel=OPT-6.7B, Number of training samples=1,000, Optimization Method=MeZO, Fine-tuning Protocol=LoRA2026.03 | — | — | — | — | — | — | 93 |