Prefill Throughput on MiniCPM 4.1-8B
20,388.3Prefill Throughput (tok/s)Dense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 20,388.3 | |
| SparDASequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 19,845.6 | |
| SparseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 18,706.6 | |
| SparseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 18,548.4 | |
| InfiniGenSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 18,445.3 | |
| SparDASequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 18,379.5 | |
| SparDASequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 17,715.2 | |
| SparDASequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 17,087.6 | |
| SparseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 16,377.3 | |
| SparseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 16,254.4 | |
| InfiniGenSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 16,249.9 | |
| SparseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 14,774.1 | |
| InfiniGenSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 14,749.2 | |
| SparseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 14,707.7 | |
| SparseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 13,676.1 | |
| DenseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 13,673.7 | |
| SparseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 13,661.8 | |
| InfiniGenSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 13,643.9 | |
| DenseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 10,228.3 | |
| DenseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 8,085.8 |