Prefill Throughput on 96K context
7,959.4Prefill Throughput (tok/s)SparDA
Evaluation results
| Method | Links | |
|---|---|---|
| SparDABackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 7,959.4 | |
| SparseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 6,776.8 | |
| SparseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 6,765.9 | |
| InfiniGenBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 6,740 | |
| SparDABackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,651.5 | |
| DenseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 5,313.5 | |
| DenseBackbone=MiniCPM4.1-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 5,260.2 | |
| SparseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=false2026.06 | 5,022.4 | |
| SparseBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,021.9 | |
| InfiniGenBackbone=NOSA-8B, Hardware=NVIDIA A100 GPU, Batch Size=4, KV-offloading=true2026.06 | 5,005.2 |