Text Classification on 20 Newsgroups Dir(0.5) (test)
70.93AccuracyFL (w/o LoRA)
Evaluation results
| Method | Links | |
|---|---|---|
| FL (w/o LoRA)Backbone=RoBERTa-base, Communicated Parameters=186B2024.10 | 70.93 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.99B2024.10 | 70.59 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.99B2024.10 | 70.44 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.991B2024.10 | 70.33 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.270B2024.10 | 70.31 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.888B2024.10 | 70.28 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.31B2024.10 | 70.26 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.497B2024.10 | 70.14 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.528B2024.10 | 70.12 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.991B2024.10 | 70.05 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.497B2024.10 | 70.03 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.249B2024.10 | 69.53 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.249B2024.10 | 69.4 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.249B2024.10 | 68.12 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.497B2024.10 | 67.75 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.124B2024.10 | 67.73 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=0.991B2024.10 | 67 |