Natural Language Understanding on GLUE (test) (SST-2, QNLI, QQP, RTE, MNLI)
95.8SST-2 ScoreFedRot-LoRA
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FedRot-LoRAClients Num=10, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 95.8 | 92.1 | 86.8 | 79.8 | 86.4 | 88.18 | |
| RoLoRAClients Num=10, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 95.6 | 91.1 | 86.3 | 80.5 | 85.8 | 87.86 | |
| FedRot-LoRAClients Num=3, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 95.4 | 92.6 | 84.2 | 86.8 | 87.6 | 89.32 | |
| FedITClients Num=3, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 95.3 | 92.6 | 77.1 | 84 | 86.6 | 87.12 | |
| RoLoRAClients Num=3, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 95.1 | 91.7 | 84.1 | 85.4 | 86.8 | 88.62 | |
| FlexLoRAClients Num=3, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 94.9 | 91 | 83.2 | 81.3 | 84.5 | 86.98 | |
| FedITClients Num=10, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 94.9 | 91.3 | 84.9 | 62.1 | 84.9 | 83.62 | |
| FedRot-LoRAClients Num=50, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 94.7 | 92 | 85.4 | 78.7 | 85.9 | 87.34 | |
| FFA-LoRAClients Num=10, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 94.6 | 89.4 | 84.9 | 60 | 84 | 82.58 | |
| FFA-LoRAClients Num=50, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 93.3 | 85.1 | 71.6 | 65.8 | 70.1 | 77.18 | |
| FedITClients Num=50, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 92.8 | 80.3 | 81.4 | 62.2 | 67.3 | 76.8 | |
| FlexLoRAClients Num=10, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 92.2 | 87.6 | 80.1 | 59.2 | 79.2 | 79.66 | |
| RoLoRAClients Num=50, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 85.1 | 89.3 | 84.7 | 73.3 | 79.8 | 82.44 | |
| FFA-LoRAClients Num=3, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 76.8 | 92.6 | 83.8 | 83 | 86.2 | 84.48 | |
| FlexLoRAClients Num=50, Rank (r)=4, Dirichlet concentration parameter (h)=0.5, Backbone=RoBERTa-Large2026.02 | 75.2 | 73.3 | 73.9 | 63.2 | 67.7 | 70.66 |