List your company
ResearchFormerly WizWand

Training Speed Measurement on LLM Sequences (train)

13.4Step Time (ms)HSAP
-243.151,488.56253,220.2754,951.9875Jun 29, 2026
Updated 4mo ago

Evaluation results

MethodLinks
HSAPSequence Per GPU=4K, GPUs=82026.0613.4
Deepspeed-UlyssesSequence Per GPU=4K, GPUs=82026.0614.6
DistFlashAttnSequence Per GPU=4K, GPUs=82026.0620.46
HSAPSequence Per GPU=4K, GPUs=162026.0626.87
Deepspeed-UlyssesSequence Per GPU=4K, GPUs=162026.0629.31
HSAPSequence Per GPU=8K, GPUs=82026.0645.88
Deepspeed-UlyssesSequence Per GPU=8K, GPUs=82026.0649.74
DistFlashAttnSequence Per GPU=4K, GPUs=162026.0654.58
DistFlashAttnSequence Per GPU=8K, GPUs=82026.0660.57
HSAPSequence Per GPU=8K, GPUs=162026.0692.54
Deepspeed-UlyssesSequence Per GPU=8K, GPUs=162026.06100.42
HSAPSequence Per GPU=4K, GPUs=642026.06116.68
DistFlashAttnSequence Per GPU=8K, GPUs=162026.06136.99
HSAPSequence Per GPU=16K, GPUs=82026.06168.44
Deepspeed-UlyssesSequence Per GPU=16K, GPUs=82026.06176.23
DistFlashAttnSequence Per GPU=16K, GPUs=82026.06205.29
DistFlashAttnSequence Per GPU=4K, GPUs=642026.06282.82
HSAPSequence Per GPU=16K, GPUs=162026.06335.71
Deepspeed-UlyssesSequence Per GPU=16K, GPUs=162026.06349.85
HSAPSequence Per GPU=8K, GPUs=642026.06436.45
DistFlashAttnSequence Per GPU=16K, GPUs=162026.06436.65
DistFlashAttnSequence Per GPU=8K, GPUs=642026.06613.12
HSAPSequence Per GPU=32K, GPUs=82026.06662.98
Deepspeed-UlyssesSequence Per GPU=32K, GPUs=82026.06664.33
DistFlashAttnSequence Per GPU=32K, GPUs=82026.06763.13
HSAPSequence Per GPU=32K, GPUs=162026.061,350.28
Deepspeed-UlyssesSequence Per GPU=32K, GPUs=162026.061,367.85
DistFlashAttnSequence Per GPU=32K, GPUs=162026.061,545.35
HSAPSequence Per GPU=16K, GPUs=642026.061,645.43
DistFlashAttnSequence Per GPU=16K, GPUs=642026.061,788.33
HSAPSequence Per GPU=32K, GPUs=642026.066,156.32
DistFlashAttnSequence Per GPU=32K, GPUs=642026.066,427.15