Natural Language Understanding on GLUE (SST-2, QNLI, QQP, MNLI Subset)
95.64SST-2RoBERTa-Large (FedTT+)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| RoBERTa-Large (FedTT+)Data Distribution=sever het., Backbone=RoBERTa-Large, # Param.=0.03M2024.10 | 95.64 | 91.67 | 86.66 | 87.73 | 90.42 | |
| RoBERTa-Large (FedTT+)Data Distribution=mild het., Backbone=RoBERTa-Large, # Param.=0.03M2024.10 | 95.53 | 93.54 | 87.91 | 88.45 | 91.36 | |
| RoBERTa-Large (FedTT+)Data Distribution=i.i.d., Backbone=RoBERTa-Large, # Param.=0.03M2024.10 | 95.41 | 94.05 | 88.15 | 88.65 | 91.56 | |
| RoBERTa-Large (FFA-LoRA r=8)Data Distribution=i.i.d., Backbone=RoBERTa-Large, # Param.=0.79M2024.10 | 95.14 | 92.64 | 86.31 | 87.13 | 90.3 | |
| RoBERTa-Large (LoRA r=8)Data Distribution=i.i.d., Backbone=RoBERTa-Large, # Param.=1.57M2024.10 | 94.42 | 91.38 | 84.47 | 86.9 | 89.29 | |
| RoBERTa-Large (LoRA r=8)Data Distribution=sever het., Backbone=RoBERTa-Large, # Param.=1.57M2024.10 | 94.32 | 88.95 | 83.51 | 82.03 | 87.2 | |
| RoBERTa-Large (FFA-LoRA r=8)Data Distribution=sever het., Backbone=RoBERTa-Large, # Param.=0.79M2024.10 | 94.32 | 90.35 | 84.35 | 85.05 | 88.52 | |
| RoBERTa-Large (FFA-LoRA r=8)Data Distribution=mild het., Backbone=RoBERTa-Large, # Param.=0.79M2024.10 | 94.1 | 91.62 | 85.33 | 87.04 | 89.52 | |
| RoBERTa-Large (LoRA r=8)Data Distribution=mild het., Backbone=RoBERTa-Large, # Param.=1.57M2024.10 | 93.55 | 91.36 | 84.41 | 87.01 | 89.08 |