Communication Efficiency Analysis on Transformer Communication Workload
99.8Communication Overhead ReductionFlash-TPSP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Flash-TPSPSequence Length=4K2026.04 | 99.8 | 36.9 | |
| Flash-TPSPSequence Length=8K2026.04 | 99.8 | 35 | |
| Flash-TPSPSequence Length=16K2026.04 | 99.8 | 38.8 | |
| Flash-TPSPSequence Length=1K2026.04 | 99.7 | 35.8 | |
| Flash-TPSPSequence Length=2K2026.04 | 99.7 | 40.2 | |
| Data SlicingSequence Length=16K2026.04 | 85.2 | 33.5 | |
| Data SlicingSequence Length=4K2026.04 | 83.9 | 31.4 | |
| Data SlicingSequence Length=8K2026.04 | 82.7 | 29.5 | |
| Data SlicingSequence Length=2K2026.04 | 78.9 | 35 | |
| Data SlicingSequence Length=1K2026.04 | 77.9 | 29.9 |