Language Modeling on WikiText-2 GPT-2-small
37.05PerplexityFedAvg
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FedAvgK=4, Trounds=5, Elocal=1, m=2,048, clip=±402026.05 | 37.05 | 8.36 | |
| FedDFQuantization status=no quantization, K=4, Trounds=5, Elocal=1, m=2,048, clip=±402026.05 | 45.41 | 0 | |
| PretrainedFine-tuning status=no fine-tune, K=4, Trounds=5, Elocal=1, m=2,048, clip=±402026.05 | 59.15 | 13.74 | |
| FPLDnbits=8, K=4, Trounds=5, Elocal=1, m=2,048, clip=±402026.05 | 61.01 | 15.6 |