Instruction Following Speed Metrics on MT-Bench
47.1Throughput (TPS)MoE-SpAc
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MoE-SpAcBase Model=DeepSeek-V2-Lite2026.02 | 47.1 | 12.66 | |
| llama.cpp-w/ SDBase Model=DeepSeek-V2-Lite, Speculative Decoding=true2026.02 | 30.78 | 21.83 | |
| HybriMoEBase Model=DeepSeek-V2-Lite2026.02 | 18.43 | 33.32 | |
| LeverTarget Model=Qwen3-8B, Draft Model=EAGLE32026.05 | 1.516 | — | |
| EdgeLLMTarget Model=Qwen3-8B, Draft Model=EAGLE32026.05 | 1.283 | — | |
| LeverTarget Model=Qwen3-8B, Draft Model=Qwen3 (0.6B)2026.05 | 1.11 | — | |
| Chain-SDTarget Model=Qwen3-8B, Draft Model=EAGLE32026.05 | 1.108 | — | |
| EdgeLLMTarget Model=Qwen3-8B, Draft Model=Qwen3 (0.6B)2026.05 | 0.89 | — | |
| Chain-SDTarget Model=Qwen3-8B, Draft Model=Qwen3 (0.6B)2026.05 | 0.82 | — |