Prefill Throughput on Context Length 32K
9,576.2Prefill Throughput (tok/s)Dense
Evaluation results
| Method | Links | |
|---|---|---|
| DenseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 9,576.2 | |
| DenseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 9,531.5 | |
| SparDABackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 8,555.9 | |
| SparseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 8,052.3 | |
| SparseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 8,002.2 | |
| InfiniGenBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 7,985.5 | |
| SparDABackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 6,097.4 | |
| SparseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,839.7 | |
| SparseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 5,833.2 | |
| InfiniGenBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,815.4 |