Text Classification on 20 Newsgroups (test)
85.8AccuracyFull Fine-tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Full Fine-tuningParams (%)=100%, Scenario=Non-Fed2024.11 | 85.8 | — | |
| SFMalpha=52025.11 | 85.3 | — | |
| AdaMixParams (%)=0.559%, Scenario=Non-Fed2024.11 | 84.7 | — | |
| SFMalpha=12025.11 | 84.7 | — | |
| SFMalpha=0.12025.11 | 84.7 | — | |
| LoRAParams (%)=0.111%, Scenario=Non-Fed2024.11 | 84.3 | — | |
| AdapterParams (%)=0.455%, Scenario=Non-Fed2024.11 | 84 | — | |
| C2AParams (%)=0.097%, Scenario=Non-Fed2024.11 | 83.9 | — | |
| CompacterParams (%)=0.043%, Scenario=Non-Fed2024.11 | 83.2 | — | |
| BitFitParams (%)=0.078%, Scenario=Non-Fed2024.11 | 82.8 | — | |
| Full Fine-tuningParams (%)=100%, Scenario=Federated, beta (β)=52024.11 | 77.6 | — | |
| Full Fine-tuningParams (%)=100%, Scenario=Federated, beta (β)=12024.11 | 77.2 | — | |
| Prompt-tuningParams (%)=0.024%, Scenario=Non-Fed2024.11 | 74.2 | — | |
| C2AParams (%)=0.097%, Scenario=Federated, beta (β)=52024.11 | 71.6 | — | |
| C2Aalpha=52025.11 | 71.6 | — | |
| C2AParams (%)=0.097%, Scenario=Federated, beta (β)=12024.11 | 70.4 | — | |
| C2Aalpha=12025.11 | 70.4 | — | |
| LoRAParams (%)=0.111%, Scenario=Federated, beta (β)=52024.11 | 69.5 | — | |
| LoRAalpha=52025.11 | 69.5 | — | |
| AdapterParams (%)=0.455%, Scenario=Federated, beta (β)=52024.11 | 69.1 | — | |
| Adapteralpha=52025.11 | 69.1 | — | |
| AdaMixParams (%)=0.559%, Scenario=Federated, beta (β)=52024.11 | 68.7 | — | |
| AdaMixalpha=52025.11 | 68.7 | — | |
| CALSstrategy=ALM2022.11 | 68.32 | 2.04 | |
| CPC2022.11 | 68.27 | 9.46 | |
| MbLS2022.11 | 67.89 | 5.4 | |
| MbLS2021.11 | 67.89 | 5.4 | |
| LoRAParams (%)=0.111%, Scenario=Federated, beta (β)=12024.11 | 67.7 | — | |
| LoRAalpha=12025.11 | 67.7 | — | |
| LS2022.11 | 67.14 | 8.07 | |
| LS2021.11 | 67.14 | 8.07 | |
| BitFitParams (%)=0.078%, Scenario=Federated, beta (β)=52024.11 | 67.1 | — | |
| BitFitalpha=52025.11 | 67.1 | — | |
| CE2022.11 | 67.01 | 22.75 | |
| CE2021.11 | 67.01 | 22.75 | |
| Full Fine-tuningParams (%)=100%, Scenario=Federated, beta (β)=0.12024.11 | 66.8 | — | |
| BitFitParams (%)=0.078%, Scenario=Federated, beta (β)=12024.11 | 66.5 | — | |
| BitFitalpha=12025.11 | 66.5 | — | |
| ECP2022.11 | 66.48 | 22.97 | |
| ECP2021.11 | 66.48 | 22.97 | |
| MMCE2022.11 | 66.23 | 23.02 | |
| FL2022.11 | 66.08 | 10.8 | |
| FL2021.11 | 66.08 | 10.8 | |
| CompacterParams (%)=0.043%, Scenario=Federated, beta (β)=52024.11 | 65.9 | — | |
| Compacteralpha=52025.11 | 65.9 | — | |
| FLSD2022.11 | 65.85 | 10.87 | |
| FLSD2021.11 | 65.85 | 10.87 | |
| AdapterParams (%)=0.455%, Scenario=Federated, beta (β)=12024.11 | 65.5 | — | |
| Adapteralpha=12025.11 | 65.5 | — | |
| AdaMixParams (%)=0.559%, Scenario=Federated, beta (β)=12024.11 | 65.3 | — | |
| AdaMixalpha=12025.11 | 65.3 | — | |
| CompacterParams (%)=0.043%, Scenario=Federated, beta (β)=12024.11 | 62.8 | — | |
| Compacteralpha=12025.11 | 62.8 | — | |
| C2AParams (%)=0.097%, Scenario=Federated, beta (β)=0.12024.11 | 61 | — | |
| C2Aalpha=0.12025.11 | 61 | — | |
| LoRAParams (%)=0.111%, Scenario=Federated, beta (β)=0.12024.11 | 56.6 | — | |
| LoRAalpha=0.12025.11 | 56.6 | — | |
| AdapterParams (%)=0.455%, Scenario=Federated, beta (β)=0.12024.11 | 56.1 | — | |
| Adapteralpha=0.12025.11 | 56.1 | — | |
| BitFitParams (%)=0.078%, Scenario=Federated, beta (β)=0.12024.11 | 55.1 | — | |
| BitFitalpha=0.12025.11 | 55.1 | — | |
| AdaMixParams (%)=0.559%, Scenario=Federated, beta (β)=0.12024.11 | 54.5 | — | |
| AdaMixalpha=0.12025.11 | 54.5 | — | |
| Prompt-tuningParams (%)=0.024%, Scenario=Federated, beta (β)=52024.11 | 51.6 | — | |
| Prompt-tuningalpha=52025.11 | 51.6 | — | |
| CompacterParams (%)=0.043%, Scenario=Federated, beta (β)=0.12024.11 | 50.1 | — | |
| Compacteralpha=0.12025.11 | 50.1 | — | |
| Mosaicω (heterogeneity level)=0.12025.05 | 49.62 | — | |
| PA3Fedω (heterogeneity level)=0.12025.05 | 48.5 | — | |
| Prompt-tuningParams (%)=0.024%, Scenario=Federated, beta (β)=12024.11 | 46.4 | — | |
| Prompt-tuningalpha=12025.11 | 46.4 | — | |
| FedProxω (heterogeneity level)=0.12025.05 | 46.1 | — | |
| FedAvgω (heterogeneity level)=0.12025.05 | 44.95 | — | |
| Mosaicω (heterogeneity level)=0.012025.05 | 44.55 | — | |
| PA3Fedω (heterogeneity level)=0.012025.05 | 42.5 | — | |
| FedProxω (heterogeneity level)=0.012025.05 | 41.25 | — | |
| FedAvgω (heterogeneity level)=0.012025.05 | 39.81 | — | |
| Prompt-tuningParams (%)=0.024%, Scenario=Federated, beta (β)=0.12024.11 | 28.2 | — | |
| Prompt-tuningalpha=0.12025.11 | 28.2 | — |