List your company
ResearchFormerly WizWand

Prefill Throughput on MiniCPM 4.1-8B

20,388.3Prefill Throughput (tok/s)Dense
7,593.710,915.37514,237.0517,558.725Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DenseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0620,388.3
SparDASequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0619,845.6
SparseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0618,706.6
SparseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0618,548.4
InfiniGenSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0618,445.3
SparDASequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0618,379.5
SparDASequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0617,715.2
SparDASequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0617,087.6
SparseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0616,377.3
SparseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0616,254.4
InfiniGenSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0616,249.9
SparseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0614,774.1
InfiniGenSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0614,749.2
SparseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0614,707.7
SparseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0613,676.1
DenseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0613,673.7
SparseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0613,661.8
InfiniGenSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0613,643.9
DenseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0610,228.3
DenseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.068,085.8