LLM Training on Qwen2.5-7B 256K context (train)
1,301.6Throughput (tokens/sec)OOMB (Sparse Attn)
Evaluation results
| Method | Links | |
|---|---|---|
| OOMB (Sparse Attn)Hardware=1xH200, Attention Type=Sparse, Tensor Parallel=false2026.02 | 1,301.6 | |
| OOMB (Sparse Attn)Hardware=4xH200, Attention Type=Sparse, Tensor Parallel=true2026.02 | 1,265.59 | |
| OOMB (Dense Attn)Hardware=1xH200, Attention Type=Dense, Tensor Parallel=false2026.02 | 266.13 | |
| OOMB (Dense Attn)Hardware=4xH200, Attention Type=Dense, Tensor Parallel=true2026.02 | 261.47 | |
| RFAHardware=4xH2002026.02 | 218.41 | |
| Ring Attn (report)Hardware=8xA1002026.02 | 50 | |
| Ring Attn (report)Hardware=16xTPUv42026.02 | 49 |