Image Classification on CIFAR-10 ResNet20 (test)
91.95AccuracySystem-aware Biased Local SGD
Evaluation results
| Method | Links | |
|---|---|---|
| System-aware Biased Local SGDEpochs=300, Comm. per epoch=4, Wall-clock Time (sec)=121.72, Hardware=2-CPU + 8-GPU, Backbone=ResNet202025.08 | 91.95 | |
| Synchronous SGD + Data-ParallelismEpochs=100, Comm. per epoch=160, Wall-clock Time (sec)=1234.75, Hardware=8-GPU only, Backbone=ResNet202025.08 | 91.28 | |
| System-aware Biased Local SGDEpochs=200, Comm. per epoch=4, Wall-clock Time (sec)=81.15, Hardware=2-CPU + 8-GPU, Backbone=ResNet202025.08 | 91.26 | |
| BytePSEpochs=100, Comm. per epoch=160, Wall-clock Time (sec)=1279.43, Hardware=2-CPU + 8-GPU, Backbone=ResNet202025.08 | 90.99 | |
| Synchronous SGD + Data-ParallelismEpochs=100, Comm. per epoch=128, Wall-clock Time (sec)=1298.39, Hardware=2-CPU + 8-GPU, Backbone=ResNet202025.08 | 90.91 | |
| System-aware Biased Local SGDEpochs=100, Comm. per epoch=4, Wall-clock Time (sec)=40.57, Hardware=2-CPU + 8-GPU, Backbone=ResNet202025.08 | 89.16 | |
| Balanced Local SGDEpochs=100, Comm. per epoch=4, Wall-clock Time (sec)=548.04, Hardware=2-CPU + 8-GPU, Backbone=ResNet202025.08 | 88.93 | |
| DLBEpochs=100, Comm. per epoch=128, Wall-clock Time (sec)=987.8, Hardware=2-CPU + 8-GPU, Backbone=ResNet202025.08 | 86.89 |