Language Understanding on MMLU-Pro (TPS and Latency)
25.8TPSMoE-SpAc
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MoE-SpAc2026.02 | 25.8 | 23.11 | |
| llama.cppspeculative decoding=true2026.02 | 17.29 | 32.88 | |
| llama.cppspeculative decoding=false2026.02 | 15.17 | 37.92 | |
| HybriMoE2026.02 | 12.22 | 47.62 | |
| Fate2026.02 | 7.93 | 73.65 | |
| SP-MoE2026.02 | 4.75 | 116.73 | |
| MoE-Infinity2026.02 | 3.64 | 151.43 | |
| vLLM2026.02 | 2.65 | 203.74 | |
| Mixtral Offload2026.02 | 2.43 | 230.87 | |
| Accelerate2026.02 | 1.13 | 495.37 |