Text Classification on BANKING77 Dir(0.01) (test)
85.98AccuracyLoRA-A2
Evaluation Results
| Method | Links | |
|---|---|---|
| LoRA-A2# of Ranks=8, Backbone=ROBERTa-Large2024.10 | 85.98 | |
| LoRA-A2# of Ranks=1, Backbone=ROBERTa-Large2024.10 | 85.66 | |
| LoRA-A2# of Ranks=4, Backbone=ROBERTa-Large2024.10 | 84.62 | |
| LoRA-A2# of Ranks=2, Backbone=ROBERTa-Large2024.10 | 83.4 | |
| FL+LoRA# of Ranks=8, Backbone=ROBERTa-Large2024.10 | 80.15 | |
| FL+LoRA# of Ranks=4, Backbone=ROBERTa-Large2024.10 | 78.97 | |
| FL+LoRA# of Ranks=2, Backbone=ROBERTa-Large2024.10 | 75.09 | |
| FL+LoRA# of Ranks=1, Backbone=ROBERTa-Large2024.10 | 73.75 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.31B2024.10 | 70.13 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.99B2024.10 | 69.84 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.888B2024.10 | 69.63 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.528B2024.10 | 69.4 | |
| LoRA-A2Backbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.270B2024.10 | 68.88 | |
| FL (w/o LoRA)Backbone=RoBERTa-base, Communicated Parameters=186B2024.10 | 67.58 | |
| FFA-LoRA# of Ranks=8, Backbone=ROBERTa-Large2024.10 | 62.98 | |
| FFA-LoRA# of Ranks=4, Backbone=ROBERTa-Large2024.10 | 62.45 | |
| FFA-LoRA# of Ranks=2, Backbone=ROBERTa-Large2024.10 | 61.55 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.99B2024.10 | 60.96 | |
| FFA-LoRA# of Ranks=1, Backbone=ROBERTa-Large2024.10 | 58.06 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.991B2024.10 | 57.38 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.497B2024.10 | 55.24 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.991B2024.10 | 54.99 | |
| LoRA-A2Backbone=DistilBERT, Number of Ranks=8, Heterogeneity level=Dir(0.01)2024.10 | 52.97 | |
| LoRA-A2Backbone=DistilBERT, Number of Ranks=4, Heterogeneity level=Dir(0.01)2024.10 | 51.24 | |
| FlexLoRABackbone=DistilBERT, Number of Ranks=8, Heterogeneity level=Dir(0.01)2024.10 | 51.21 | |
| LoRA-A2Backbone=DistilBERT, Number of Ranks=2, Heterogeneity level=Dir(0.01)2024.10 | 49.97 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.497B2024.10 | 49.08 | |
| LoRA-A2Backbone=DistilBERT, Number of Ranks=1, Heterogeneity level=Dir(0.01)2024.10 | 48.89 | |
| FL + LoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.249B2024.10 | 45.78 | |
| FlexLoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.249B2024.10 | 42.75 | |
| FlexLoRABackbone=DistilBERT, Number of Ranks=4, Heterogeneity level=Dir(0.01)2024.10 | 41.26 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=0.991B2024.10 | 40.88 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.497B2024.10 | 37.63 | |
| FL+LoRABackbone=DistilBERT, Number of Ranks=4, Heterogeneity level=Dir(0.01)2024.10 | 36.92 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.249B2024.10 | 34.44 | |
| FlexLoRABackbone=DistilBERT, Number of Ranks=2, Heterogeneity level=Dir(0.01)2024.10 | 34.05 | |
| FFA-LoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.124B2024.10 | 33.68 | |
| FL+LoRABackbone=DistilBERT, Number of Ranks=8, Heterogeneity level=Dir(0.01)2024.10 | 32.58 | |
| FL+LoRABackbone=DistilBERT, Number of Ranks=2, Heterogeneity level=Dir(0.01)2024.10 | 27.14 | |
| FL+LoRABackbone=DistilBERT, Number of Ranks=1, Heterogeneity level=Dir(0.01)2024.10 | 21.59 | |
| FlexLoRABackbone=DistilBERT, Number of Ranks=1, Heterogeneity level=Dir(0.01)2024.10 | 21.01 | |
| FFA-LoRABackbone=DistilBERT, Number of Ranks=8, Heterogeneity level=Dir(0.01)2024.10 | 18.82 | |
| FFA-LoRABackbone=DistilBERT, Number of Ranks=4, Heterogeneity level=Dir(0.01)2024.10 | 16.73 | |
| FFA-LoRABackbone=DistilBERT, Number of Ranks=2, Heterogeneity level=Dir(0.01)2024.10 | 15.49 | |
| FFA-LoRABackbone=DistilBERT, Number of Ranks=1, Heterogeneity level=Dir(0.01)2024.10 | 14.29 |