Inference Throughput on 64K scenario
8,191.3Prefill Throughput (tok/s)SparDA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SparDABackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 8,191.3 | — | — | |
| SparseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 7,311 | — | — | |
| SparseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 7,310.1 | — | — | |
| InfiniGenBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 7,268.6 | — | — | |
| DenseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 6,870.5 | — | — | |
| DenseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 6,836.1 | — | — | |
| SparDABackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,817.9 | — | — | |
| SparseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,365 | — | — | |
| SparseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 5,364.6 | — | — | |
| InfiniGenBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,339 | — | — | |
| gpt-oss-puzzle-88BHardware=8x B200, Scenario=64K/64K, Note=Max throughput at best configuration2026.02 | — | — | 1.4 |