Language Modeling on Pile uncopyrighted (test)
3.608Worst Log-PerplexityFGDRO-KL-Adam
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FGDRO-KL-AdamModel=GPT2, Local iterations=200K, Batch size=322024.10 | 3.608 | 2.653 | |
| FedAdamModel=GPT2, Local iterations=200K, Batch size=322024.10 | 7.242 | 6.479 | |
| FGDRO-KLModel=GPT2, Local iterations=200K, Batch size=322024.10 | 7.932 | 6.664 | |
| SCAFFOLDModel=GPT2, Local iterations=200K, Batch size=322024.10 | 7.975 | 6.901 | |
| DRFAModel=GPT2, Local iterations=200K, Batch size=322024.10 | 8.014 | 6.702 | |
| DR-DSGDModel=GPT2, Local iterations=200K, Batch size=322024.10 | 8.023 | 6.693 | |
| FedProxModel=GPT2, Local iterations=200K, Batch size=322024.10 | 8.079 | 6.724 | |
| FedAvgModel=GPT2, Local iterations=200K, Batch size=322024.10 | 8.085 | 6.785 | |
| FGDRO-CVaRModel=GPT2, Local iterations=200K, Batch size=322024.10 | 8.145 | 6.907 |