Language Modeling on BookCorpus (val)
17.81PerplexityDec. CP Comp.
Evaluation results
| Method | Links | |
|---|---|---|
| Dec. CP Comp.Bandwidth=300Mbps, TPS=55K (×20), MEM (GB)/GPU=38.7 (+0.7%), Context Window=132K, Training Tokens=16B2026.06 | 17.81 | |
| Cen. CPBandwidth=100Gbps, TPS=56K, MEM (GB)/GPU=38.4, Context Window=132K, Training Tokens=16B2026.06 | 17.88 | |
| AsyncMeshMesh Configuration=4 x 2, Iterations=30k, AsyncPP=true2026.01 | 35.09 | |
| SPARTAMesh Configuration=4 x 2, Iterations=30k, AsyncPP=true2026.01 | 35.15 | |
| DPAvgMesh Configuration=4 x 2, Iterations=30k, AsyncPP=true2026.01 | 35.24 | |
| FullSyncMesh Configuration=4 x 2, Iterations=30k, AsyncPP=false2026.01 | 35.99 | |
| AsyncSPARTAMesh Configuration=4 x 2, Iterations=30k, AsyncPP=true2026.01 | 37.83 |