List your company
ResearchFormerly WizWand

LLM Decoding Throughput on MiniCPM4.1-8B (32K context)

1,358.2Throughput (tok/s)SparDA
4.224355.737707.251,058.763Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SparDABatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.061,358.2
SparDABatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,328
SparseBatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.061,225.1
SparDABatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.061,205.2
SparseBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,088.4
SparseBatch size=16, Offloading=Disabled, Hardware=NVIDIA H1002026.061,073
SparDABatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06918.5
SparseBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06861.8
SparseBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06661.5
SparseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06550.9
SparDABatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06471.9
DenseBatch size=16, Offloading=Disabled, Hardware=NVIDIA H1002026.06440.2
SparseBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06420.9
DenseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06352.2
SparseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06277.2
DenseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06248.3
SparDABatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06235.7
SparseBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06224.4
InfiniGenBatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.06222.4
InfiniGenBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.06184.8
InfiniGenBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06140.4
InfiniGenBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.0698.8
InfiniGenBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.0675.1
InfiniGenBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.0656.3