Natural Language Understanding on GLUE v1.0 (test) (Subset: QNLI, MNLI, SST-2, QQP, SNLI)
81.06QNLI AccuracyAS-LoRA
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| AS-LoRADP Budget (ϵ)=32026.05 | 81.06 | 79.15 | 80.03 | 92.86 | 78.51 | 85.23 | 82.81 | |
| FedLoRADP Budget (ϵ)=32026.05 | 72.43 | 63.76 | 64.98 | 91.37 | 67.86 | 78.19 | 73.1 | |
| FFA-LoRADP Budget (ϵ)=12026.05 | 71.95 | 62.7 | 65.04 | 79.27 | 71 | 81.37 | 71.89 | |
| AS-LoRADP Budget (ϵ)=12026.05 | 71.52 | 76.71 | 77.54 | 80.76 | 74.94 | 84.32 | 77.63 | |
| FFA-LoRADP Budget (ϵ)=32026.05 | 68.18 | 65.06 | 67.51 | 91 | 75.79 | 77.91 | 74.24 | |
| RoLoRADP Budget (ϵ)=32026.05 | 65.02 | 69.15 | 71.02 | 91.37 | 72.86 | 82.22 | 75.27 | |
| FedLoRADP Budget (ϵ)=12026.05 | 55.72 | 58.03 | 59.37 | 57.57 | 65.72 | 74.79 | 61.87 | |
| RoLoRADP Budget (ϵ)=12026.05 | 53.85 | 59.91 | 61.37 | 62.99 | 73.1 | 80.97 | 65.37 |