Text Summarization on CNN/DM (TPS, Latency)
50.17TPSMoE-SpAc
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MoE-SpAcBase Model=DeepSeek-V2-Lite2026.02 | 50.17 | 8.13 | |
| llama.cpp-w/ SDBase Model=DeepSeek-V2-Lite, Speculative Decoding=true2026.02 | 33.23 | 15.94 | |
| MoE-SpAc2026.02 | 22.78 | 14.31 | |
| HybriMoEBase Model=DeepSeek-V2-Lite2026.02 | 18.43 | 21.78 | |
| llama.cppspeculative decoding=true2026.02 | 16.46 | 18.36 | |
| llama.cppspeculative decoding=false2026.02 | 15.42 | 19.46 | |
| HybriMoE2026.02 | 12.37 | 25.11 | |
| Fate2026.02 | 7.83 | 40.78 | |
| SP-MoE2026.02 | 4.38 | 65.24 | |
| MoE-Infinity2026.02 | 3.56 | 80.56 | |
| vLLM2026.02 | 2.57 | 104.28 | |
| Mixtral Offload2026.02 | 2.23 | 131.72 | |
| Accelerate2026.02 | 1.27 | 222.47 |