Text Classification on 20 Newsgroups Dir(0.01) (test)
0.6441AccuracyFL (w/o LoRA)
Evaluation Results
| Method | Links | |
|---|---|---|
| FL (w/o LoRA)Backbone=RoBERTa-base, Communicated Parameters=186B2024.10 | 0.6441 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.99B2024.10 | 0.6041 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.270B2024.10 | 0.5484 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.31B2024.10 | 0.545 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.888B2024.10 | 0.5304 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.528B2024.10 | 0.5299 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.991B2024.10 | 0.505 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.497B2024.10 | 0.4823 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.99B2024.10 | 0.439 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.991B2024.10 | 0.4312 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.497B2024.10 | 0.3907 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=0.991B2024.10 | 0.3734 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.249B2024.10 | 0.3665 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.249B2024.10 | 0.3658 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.497B2024.10 | 0.3604 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.249B2024.10 | 0.3554 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.124B2024.10 | 0.3444 |