List your company
ResearchFormerly WizWand

LLM Decoding Throughput on NOSA-8B 128K context length

1,463.3Throughput (tok/s)SparDA
22.068396.234770.41,144.566Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SparDABatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,463.3
SparseBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,298.3
SparDABatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.061,127
SparseBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06898.7
SparDABatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06735
SparseBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06529.2
SparDABatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06399.3
SparseBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06285.4
SparDABatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06219
SparseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06179.3
SparseBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06173.2
InfiniGenBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06166.9
InfiniGenBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06131.7
DenseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06108.4
InfiniGenBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06105.2
InfiniGenBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.0677.5