List your company
ResearchFormerly WizWand

LLM Decoding Throughput on NOSA-8B 64K context length

1,996.3Throughput (tok/s)SparDA
1.372519.2861,037.21,555.114Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SparDABatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.061,996.3
SparDABatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,905.2
SparseBatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.061,793.1
SparseBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,553.1
SparDABatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.061,392.3
SparseBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.061,071.1
SparDABatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06870.9
SparseBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06704.6
SparseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06439.7
SparDABatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06431.2
SparseBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06397.9
SparseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06233.2
SparseBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06221
InfiniGenBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.06220.4
SparDABatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06218.8
DenseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06217.6
InfiniGenBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06181
DenseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06174.3
InfiniGenBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06139.1
InfiniGenBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06106.2
InfiniGenBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.0678.1