Language Modeling on OpenWebText Regime WR (val)
52Time to Target (min)Echelon-DA
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Echelon-DA2026.06 | 52 | 25.1 | |
| Echelon-DA+SAStochastic Averaging=true2026.06 | 53 | 25.1 | |
| Central2026.06 | 60 | 25 | |
| Echelon2026.06 | 65 | 25.1 | |
| SCAFFOLD2026.06 | 72 | 25.2 | |
| FedProx2026.06 | 75 | 25.3 | |
| DiLoCo2026.06 | 78 | 25.4 | |
| FedAvg2026.06 | 82 | 25.5 | |
| DiLoCo+SAStochastic Averaging=true2026.06 | 85.5 | 25.5 |