List your company
ResearchFormerly WizWand

LLM Decoding Throughput on MiniCPM 8B 96K context length 4.1

1,076.3Decoding Throughput (tok/s)SparDA
8.324285.587562.85840.113Jun 3, 2026
Updated 4mo ago

Evaluation results

MethodLinks
SparDABatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.061,076.3
SparDABatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06939.2
SparseBatch size=128, Offloading=Enabled, Hardware=NVIDIA H1002026.06934.3
SparseBatch size=64, Offloading=Enabled, Hardware=NVIDIA H1002026.06839.5
SparDABatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06752.9
SparseBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06679.4
SparseBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.06489
SparDABatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06475.6
SparseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06401.4
SparseBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.06306
SparDABatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06238.6
SparseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06215.5
SparseBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.06185.7
DenseBatch size=8, Offloading=Disabled, Hardware=NVIDIA H1002026.06159
DenseBatch size=4, Offloading=Disabled, Hardware=NVIDIA H1002026.06133.8
InfiniGenBatch size=32, Offloading=Enabled, Hardware=NVIDIA H1002026.06117.9
InfiniGenBatch size=16, Offloading=Enabled, Hardware=NVIDIA H1002026.0686
InfiniGenBatch size=8, Offloading=Enabled, Hardware=NVIDIA H1002026.0664.9
InfiniGenBatch size=4, Offloading=Enabled, Hardware=NVIDIA H1002026.0649.4