Language Model Training Efficiency on Pajama
3,477.07ThroughputHSAP
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| HSAPContext Length=32K, Topology (intraSPxinterSP)=8x12026.06 | 3,477.07 | 0.94 | |
| HSAPContext Length=64K, Topology (intraSPxinterSP)=4x22026.06 | 3,312.26 | 0.45 | |
| HSAPContext Length=16K, Topology (intraSPxinterSP)=1x82026.06 | 2,693.44 | 1.46 | |
| PackingContext Length=32K, Topology (intraSPxinterSP)=8x12026.06 | 2,522.72 | 0.68 | |
| BatchingContext Length=32K, Topology (intraSPxinterSP)=8x12026.06 | 2,511.99 | 0.68 | |
| PackingContext Length=16K, Topology (intraSPxinterSP)=1x82026.06 | 2,005 | 1.09 | |
| BatchingContext Length=16K, Topology (intraSPxinterSP)=1x82026.06 | 2,003.36 | 1.09 | |
| PackingContext Length=64K, Topology (intraSPxinterSP)=4x22026.06 | 1,438.61 | 0.2 | |
| BatchingContext Length=64K, Topology (intraSPxinterSP)=4x22026.06 | 970.22 | 0.2 |