Natural Language Understanding on GLUE (test) (SST-2, QNLI, QQP, MNLI Subset)
95.64SST-2 AccuracyFedTT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| FedTTPriv. Budget (ε)=1, #Param.=0.24M, Backbone=RoBERTa-Large2024.10 | 95.64 | 91.67 | 83.11 | 83.81 | — | |
| LoRAPriv. Budget (ε)=1, #Param.=1.57M, rank=8, Backbone=RoBERTa-Large2024.10 | 94.32 | 88.95 | 81.28 | 33.8 | — | |
| FFA-LoRAPriv. Budget (ε)=1, #Param.=0.79M, rank=8, Backbone=RoBERTa-Large2024.10 | 94.32 | 90.35 | 82.5 | 75.05 | — | |
| FedTTPriv. Budget (ε)=3, #Param.=0.24M, Backbone=RoBERTa-Large2024.10 | 93.96 | 86.64 | 84.36 | 85.08 | — | |
| LA-LoRAPrivacy=Non-private, Backbone=RoBERTa-Base, Data Heterogeneity=Dirichlet beta = 0.32026.02 | 93.94 | 89.96 | 86.41 | 83.32 | 88.41 | |
| FedTTPriv. Budget (ε)=6, #Param.=0.24M, Backbone=RoBERTa-Large2024.10 | 93.8 | 87.43 | 84.61 | 85.45 | — | |
| FFA-LoRAPriv. Budget (ε)=6, #Param.=0.79M, rank=8, Backbone=RoBERTa-Large2024.10 | 93.73 | 87.27 | 83.31 | 78.81 | — | |
| LoRAPriv. Budget (ε)=6, #Param.=1.57M, rank=8, Backbone=RoBERTa-Large2024.10 | 93.7 | 84.99 | 82.11 | 39.46 | — | |
| RoLoRAPrivacy=Non-private, Backbone=RoBERTa-Base, Data Heterogeneity=Dirichlet beta = 0.32026.02 | 93.65 | 88.65 | 85.52 | 82.22 | 87.51 | |
| FFA-LoRAPriv. Budget (ε)=3, #Param.=0.79M, rank=8, Backbone=RoBERTa-Large2024.10 | 93.59 | 86.18 | 83.03 | 77.42 | — | |
| LA-LoRAModel=Llama-2-7B, Privacy Epsilon (epsilon)=12026.02 | 93.36 | 89.78 | 86.75 | 87.56 | 89.36 | |
| LoRAPriv. Budget (ε)=3, #Param.=1.57M, rank=8, Backbone=RoBERTa-Large2024.10 | 93.32 | 83.94 | 82.08 | 35.82 | — | |
| LA-LoRAPrivacy Budget (epsilon)=3, Backbone=RoBERTa-Base2026.02 | 93.12 | 89.83 | 85.83 | 82.99 | — | |
| LA-LoRAPrivacy Budget (epsilon)=2, Backbone=RoBERTa-Base2026.02 | 93 | 89.18 | 85.64 | 82.87 | — | |
| RoLoRAPrivacy Budget (epsilon)=3, Backbone=RoBERTa-Base2026.02 | 92.7 | 88.23 | 85.35 | 82.12 | — | |
| LA-LoRAPrivacy Budget (epsilon)=1, Backbone=RoBERTa-Base2026.02 | 92.66 | 88.73 | 85.34 | 82.35 | — | |
| FFA-LoRAPrivacy=Non-private, Backbone=RoBERTa-Base, Data Heterogeneity=Dirichlet beta = 0.32026.02 | 92.56 | 87.53 | 85.36 | 81.54 | 86.75 | |
| RoLoRAPrivacy Budget (epsilon)=2, Backbone=RoBERTa-Base2026.02 | 92.55 | 87.08 | 85.02 | 82.01 | — | |
| FFA-LoRAModel=Llama-2-7B, Privacy Epsilon (epsilon)=12026.02 | 92.53 | 89.23 | 85.56 | 86.98 | 88.58 | |
| FFA-LoRAPrivacy Budget (epsilon)=2, Backbone=RoBERTa-Base2026.02 | 92.39 | 87.3 | 84.73 | 81.94 | — | |
| DP-LoRAPrivacy Budget (epsilon)=3, Backbone=RoBERTa-Base2026.02 | 92.36 | 86.31 | 84.56 | 80.98 | — | |
| FFA-LoRAPrivacy Budget (epsilon)=3, Backbone=RoBERTa-Base2026.02 | 92.32 | 87.2 | 85.12 | 81.71 | — | |
| RoLoRAPrivacy Budget (epsilon)=1, Backbone=RoBERTa-Base2026.02 | 92.32 | 86.25 | 84.49 | 81.54 | — | |
| DP-LoRAPrivacy Budget (epsilon)=2, Backbone=RoBERTa-Base2026.02 | 92.2 | 86.03 | 84.26 | 80.62 | — | |
| RoLoRAModel=Llama-2-7B, Privacy Epsilon (epsilon)=12026.02 | 92.12 | 89.34 | 85.98 | 87.21 | 88.66 | |
| LA-LoRAPrivacy=epsilon = 1, Backbone=RoBERTa-Base, Data Heterogeneity=Dirichlet beta = 0.32026.02 | 92.11 | 87.13 | 85.04 | 82.27 | 86.64 | |
| DP-LoRAPrivacy=Non-private, Backbone=RoBERTa-Base, Data Heterogeneity=Dirichlet beta = 0.32026.02 | 92.07 | 86.25 | 84.02 | 81.22 | 85.89 | |
| RoLoRAPrivacy=epsilon = 1, Backbone=RoBERTa-Base, Data Heterogeneity=Dirichlet beta = 0.32026.02 | 91.74 | 85.86 | 84.21 | 81.35 | 85.78 | |
| DP-LoRAModel=Llama-2-7B, Privacy Epsilon (epsilon)=12026.02 | 91.56 | 88.22 | 85.56 | 86.86 | 88.05 | |
| FFA-LoRAPrivacy Budget (epsilon)=1, Backbone=RoBERTa-Base2026.02 | 91.06 | 85.08 | 84.3 | 81.14 | — | |
| DP-LoRAPrivacy Budget (epsilon)=1, Backbone=RoBERTa-Base2026.02 | 90.71 | 84.07 | 83.48 | 79.87 | — | |
| FFA-LoRAPrivacy=epsilon = 1, Backbone=RoBERTa-Base, Data Heterogeneity=Dirichlet beta = 0.32026.02 | 90.62 | 84.63 | 84.1 | 80.98 | 85.08 | |
| DP-LoRAPrivacy=epsilon = 1, Backbone=RoBERTa-Base, Data Heterogeneity=Dirichlet beta = 0.32026.02 | 90.13 | 83.79 | 83.28 | 79.8 | 84.25 |