List your company
ResearchFormerly WizWand

LLM Decoding Throughput on MiniCPM 8B 64K context length 4.1

1,159.1Throughput (tok/s)SparDA
11.044309.097607.15905.203Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SparDABatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.061,159.1
SparDABatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,049.9
SparseBatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.061,041.9
SparDABatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.061,026.5
SparseBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.06937.2
SparDABatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06811.8
SparseBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06756.1
SparseBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06549.4
SparDABatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06470.3
SparseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06466.2
SparseBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06350.8
SparseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06246.5
SparDABatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06234.9
DenseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06218.3
SparseBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06213.1
DenseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06174.1
InfiniGenBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.06166.5
InfiniGenBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06129.1
InfiniGenBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.0694
InfiniGenBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.0663.3
InfiniGenBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.0655.2