LLM Training Step Performance on Qwen3-14B S=2048 (base variant)
266.1Latency (ms)FORGE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FORGEBatch Size=1, Precision=BF16-everywhere, Flash Attention 3=true, Liger Kernel=true, Hardware=H100 SXM 80 GB, Model Variant=base variant2026.06 | 266.1 | 78.7 | |
| GaLore r=128†Batch Size=1, Precision=BF16-everywhere, Flash Attention 3=true, Liger Kernel=true, Hardware=H100 SXM 80 GB, Model Variant=base variant2026.06 | 469.5 | 66.5 | |
| APOLLO-Mini r=256†Batch Size=1, Precision=BF16-everywhere, Flash Attention 3=true, Liger Kernel=true, Hardware=H100 SXM 80 GB, Model Variant=base variant2026.06 | 541.8 | 68.3 | |
| AdaLomo†Batch Size=1, Precision=BF16-everywhere, Flash Attention 3=true, Liger Kernel=true, Hardware=H100 SXM 80 GB, Model Variant=base variant2026.06 | 1,438.4 | 53.5 |