Natural Language Understanding on GLUE i.i.d. distribution (val)
0.9268MRPCFedTT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FedTTBackbone=DeBERTa-Base, # Param.=0.06M, FL Setting=cross-silo, Distribution=i.i.d.2024.10 | 0.9268 | 0.9461 | 0.9202 | 0.884 | 0.8599 | 0.9074 | |
| FedTT+Backbone=DeBERTa-Base, # Param.=0.02M, FL Setting=cross-silo, Distribution=i.i.d.2024.10 | 0.926 | 0.9358 | 0.9054 | 0.879 | 0.8533 | 0.8999 | |
| LoRABackbone=DeBERTa-Base, # Param.=0.30M, Rank=r=8, FL Setting=cross-silo, Distribution=i.i.d.2024.10 | 0.9187 | 0.9495 | 0.9268 | 0.892 | 0.8731 | 0.911 | |
| LoRABackbone=DeBERTa-Base, # Param.=0.15M, Rank=r=4, FL Setting=cross-silo, Distribution=i.i.d.2024.10 | 0.9172 | 0.9495 | 0.9266 | 0.867 | 0.8691 | 0.9058 | |
| BitFitBackbone=DeBERTa-Base, # Param.=0.10M, FL Setting=cross-silo, Distribution=i.i.d.2024.10 | 0.9133 | 0.9472 | 0.9189 | 0.884 | 0.8602 | 0.9047 | |
| RoLoRABackbone=DeBERTa-Base, # Param.=0.08M, Rank=r=4, FL Setting=cross-silo, Distribution=i.i.d.2024.10 | 0.9117 | 0.9461 | 0.924 | 0.879 | 0.8627 | 0.9047 | |
| PromptBackbone=DeBERTa-Base, # Param.=0.01M, FL Setting=cross-silo, Distribution=i.i.d.2024.10 | 0.8296 | 0.9232 | 0.8213 | 0.805 | 0.7446 | 0.8247 | |
| P-TuningBackbone=DeBERTa-Base, # Param.=0.30M, FL Setting=cross-silo, Distribution=i.i.d.2024.10 | 0.8201 | 0.9048 | 0.8212 | 0.84 | 0.8074 | 0.8387 |