Prefill Throughput on 128K context
7,756.8Prefill Throughput (tok/s)SparDA
Evaluation Results
| Method | Links | |
|---|---|---|
| SparDABackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 7,756.8 | |
| SparseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 6,310.3 | |
| SparseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 6,299.2 | |
| InfiniGenBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 6,296.3 | |
| SparDABackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,477.5 | |
| SparseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 4,717.1 | |
| SparseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 4,713.5 | |
| InfiniGenBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 4,711.5 | |
| DenseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 4,316 | |
| DenseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 4,282.9 |