List your company
ResearchFormerly WizWand

LLM Decoding Throughput on NOSA-8B 96K context length

1,694.1Throughput (tok/s)SparDA
10.34447.47884.61,321.73Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SparDABatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,694.1
SparseBatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.061,580.5
SparseBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,354.3
SparDABatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.061,263.7
SparseBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06962.4
SparDABatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06852.9
SparseBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06605.5
SparDABatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06437.6
SparseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06380.1
SparseBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06341.3
SparDABatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06215.5
SparseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06204.1
SparseBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06191.5
InfiniGenBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06165.2
DenseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06158.3
DenseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06130
InfiniGenBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06129.1
InfiniGenBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06104
InfiniGenBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.0675.1