List your company
ResearchFormerly WizWand

Text Classification on BANKING77 Dir(0.01) (test)

85.98AccuracyLoRA-A2
11.422430.778750.13569.4913Oct 30, 2024
Updated 8mo ago

Evaluation results

MethodLinks
LoRA-A2# of Ranks=8, Backbone=ROBERTa-Large2024.1085.98
LoRA-A2# of Ranks=1, Backbone=ROBERTa-Large2024.1085.66
LoRA-A2# of Ranks=4, Backbone=ROBERTa-Large2024.1084.62
LoRA-A2# of Ranks=2, Backbone=ROBERTa-Large2024.1083.4
FL+LoRA# of Ranks=8, Backbone=ROBERTa-Large2024.1080.15
FL+LoRA# of Ranks=4, Backbone=ROBERTa-Large2024.1078.97
FL+LoRA# of Ranks=2, Backbone=ROBERTa-Large2024.1075.09
FL+LoRA# of Ranks=1, Backbone=ROBERTa-Large2024.1073.75
LoRA-A2Backbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.31B2024.1070.13
FlexLoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.99B2024.1069.84
LoRA-A2Backbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.888B2024.1069.63
LoRA-A2Backbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.528B2024.1069.4
LoRA-A2Backbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.270B2024.1068.88
FL (w/o LoRA)Backbone=RoBERTa-base, Communicated Parameters=186B2024.1067.58
FFA-LoRA# of Ranks=8, Backbone=ROBERTa-Large2024.1062.98
FFA-LoRA# of Ranks=4, Backbone=ROBERTa-Large2024.1062.45
FFA-LoRA# of Ranks=2, Backbone=ROBERTa-Large2024.1061.55
FL + LoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=1.99B2024.1060.96
FFA-LoRA# of Ranks=1, Backbone=ROBERTa-Large2024.1058.06
FlexLoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.991B2024.1057.38
FlexLoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.497B2024.1055.24
FL + LoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.991B2024.1054.99
LoRA-A2Backbone=DistilBERT, Number of Ranks=8, Heterogeneity level=Dir(0.01)2024.1052.97
LoRA-A2Backbone=DistilBERT, Number of Ranks=4, Heterogeneity level=Dir(0.01)2024.1051.24
FlexLoRABackbone=DistilBERT, Number of Ranks=8, Heterogeneity level=Dir(0.01)2024.1051.21
LoRA-A2Backbone=DistilBERT, Number of Ranks=2, Heterogeneity level=Dir(0.01)2024.1049.97
FL + LoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.497B2024.1049.08
LoRA-A2Backbone=DistilBERT, Number of Ranks=1, Heterogeneity level=Dir(0.01)2024.1048.89
FL + LoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.249B2024.1045.78
FlexLoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.249B2024.1042.75
FlexLoRABackbone=DistilBERT, Number of Ranks=4, Heterogeneity level=Dir(0.01)2024.1041.26
FFA-LoRABackbone=RoBERTa-base, LoRA Rank=8, Communicated Parameters=0.991B2024.1040.88
FFA-LoRABackbone=RoBERTa-base, LoRA Rank=4, Communicated Parameters=0.497B2024.1037.63
FL+LoRABackbone=DistilBERT, Number of Ranks=4, Heterogeneity level=Dir(0.01)2024.1036.92
FFA-LoRABackbone=RoBERTa-base, LoRA Rank=2, Communicated Parameters=0.249B2024.1034.44
FlexLoRABackbone=DistilBERT, Number of Ranks=2, Heterogeneity level=Dir(0.01)2024.1034.05
FFA-LoRABackbone=RoBERTa-base, LoRA Rank=1, Communicated Parameters=0.124B2024.1033.68
FL+LoRABackbone=DistilBERT, Number of Ranks=8, Heterogeneity level=Dir(0.01)2024.1032.58
FL+LoRABackbone=DistilBERT, Number of Ranks=2, Heterogeneity level=Dir(0.01)2024.1027.14
FL+LoRABackbone=DistilBERT, Number of Ranks=1, Heterogeneity level=Dir(0.01)2024.1021.59
FlexLoRABackbone=DistilBERT, Number of Ranks=1, Heterogeneity level=Dir(0.01)2024.1021.01
FFA-LoRABackbone=DistilBERT, Number of Ranks=8, Heterogeneity level=Dir(0.01)2024.1018.82
FFA-LoRABackbone=DistilBERT, Number of Ranks=4, Heterogeneity level=Dir(0.01)2024.1016.73
FFA-LoRABackbone=DistilBERT, Number of Ranks=2, Heterogeneity level=Dir(0.01)2024.1015.49
FFA-LoRABackbone=DistilBERT, Number of Ranks=1, Heterogeneity level=Dir(0.01)2024.1014.29