Training Throughput on Llama3 8B (train)
2,320.47Throughput (128K SeqLen)Ulysses
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| UlyssesHardware=8xH100s2026.02 | 2,320.47 | 1,503.8 | 878.63 | 475.33 | 246.05 | 162.41 | — | — | |
| UPipeHardware=8xH100s2026.02 | 2,281.05 | 1,487.29 | 867.17 | 472.53 | 246.07 | 166.32 | 125.56 | 98.25 | |
| RingHardware=8xH100s2026.02 | 2,064.9 | 1,387.67 | 841.05 | 458.51 | 237.99 | 159.96 | — | — | |
| Native PyTorchHardware=8xH100s2026.02 | 1,373.87 | 845.99 | 474.3 | 249.85 | — | — | — | — | |
| FPDTHardware=8xH100s2026.02 | 1,171.68 | 884.75 | 621.2 | 382.42 | 219.53 | 153.48 | 119.76 | — |