List your company
ResearchFormerly WizWand

LLM Decoding Throughput on MiniCPM 128K context length 4.1-8B

1,000.1Decoding Throughput (tok/s)SparDA
13.868269.909525.95781.991Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SparDABatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,000.1
SparDABatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06899.2
SparseBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.06788.9
SparDABatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06705.3
SparseBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06618.6
SparDABatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06471.2
SparseBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06447.9
SparseBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06279.5
SparDABatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06240.2
SparseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06189.5
SparseBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06167.8
InfiniGenBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06117.5
DenseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06108.6
InfiniGenBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.0685.6
InfiniGenBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.0666.5
InfiniGenBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.0651.8