List your company
ResearchFormerly WizWand

LLM Decoding Throughput on NOSA-8B 32K context length

2,312.1Throughput (tok/s)SparDA
-7.204594.9231,197.051,799.177Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SparDABatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.062,312.1
SparseBatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.062,145.5
SparDABatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.062,106.6
SparseBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,861.6
SparDABatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.061,579.9
SparseBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.061,244.9
SparseBatch size=16, Offloading=Disabled, Hardware=NVIDIA H1002026.06947.1
SparDABatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06873.2
SparseBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06795.8
SparseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06488.1
SparseBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06478
DenseBatch size=16, Offloading=Disabled, Hardware=NVIDIA H1002026.06439.7
SparDABatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06433.2
InfiniGenBatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.06356.5
DenseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06350.7
InfiniGenBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.06269.9
DenseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06252
SparseBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06244
SparseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06241.3
SparDABatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06214.7
InfiniGenBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06205.9
InfiniGenBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06166
InfiniGenBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06120.3
InfiniGenBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.0682