Code Generation on HumanEval (P@1, Spd, τ)
2.4Speed (Spd)MoE-Spec
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MoE-SpecModel=OLMoE-1B-7B, Temperature=02026.02 | 2.4 | 30 | 4.2 | |
| EAGLE-3Model=OLMoE-1B-7B, Temperature=02026.02 | 2.3 | 30 | 4.2 | |
| MoE-SpecModel=Qwen3-30B-A3B, Temperature=02026.02 | 2.2 | 78 | 3.1 | |
| EAGLE-3Model=OLMoE-1B-7B, Temperature=12026.02 | 2 | 26 | 3.6 | |
| MoE-SpecModel=OLMoE-1B-7B, Temperature=12026.02 | 2 | 29 | 3.8 | |
| MoE-SpecModel=Mixtral-8x7B, Temperature=02026.02 | 1.9 | 27 | 3.1 | |
| MoE-SpecModel=Mixtral-8x7B, Temperature=12026.02 | 1.9 | 20 | 3.1 | |
| MoE-SpecModel=Qwen3-30B-A3B, Temperature=12026.02 | 1.9 | 78 | 3 | |
| EAGLE-3Model=Qwen3-30B-A3B, Temperature=02026.02 | 1.8 | 82 | 3.1 | |
| EAGLE-3Model=Qwen3-30B-A3B, Temperature=12026.02 | 1.8 | 80 | 3 | |
| EAGLEModel=Mixtral-8x7B, Temperature=02026.02 | 1.6 | 20 | 3.2 | |
| EAGLEModel=Mixtral-8x7B, Temperature=12026.02 | 1.6 | 21 | 3.1 | |
| ARModel=Mixtral-8x7B, Temperature=02026.02 | 1 | 22 | — | |
| ARModel=OLMoE-1B-7B, Temperature=02026.02 | 1 | 30 | — | |
| ARModel=Qwen3-30B-A3B, Temperature=02026.02 | 1 | 82 | — | |
| ARModel=Mixtral-8x7B, Temperature=12026.02 | 1 | 21 | — | |
| ARModel=OLMoE-1B-7B, Temperature=12026.02 | 1 | 29 | — | |
| ARModel=Qwen3-30B-A3B, Temperature=12026.02 | 1 | 81 | — |