Code Generation on HumanEval (HumanEval Metric)
34.2HumanEvalFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullModel=Qwen1.5MoE-A2.7B, E↓=100%, A↓=100%2026.06 | 34.2 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=61.38%2026.06 | 31.1 | |
| TENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=63.76%2026.06 | 29.9 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=60%2026.06 | 28 | |
| FullModel=DeepSeek-V2-Lite, E↓=100%, A↓=100%2026.06 | 26.2 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 19.5 | |
| ENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=60%2026.06 | 19.5 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=34.51%2026.06 | 18.9 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=30%2026.06 | 17.1 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 14.6 | |
| TENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=36.38%2026.06 | 14 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 11.6 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 11.6 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 8.5 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 6.7 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 1.8 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 1.2 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 1.2 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 0 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 0 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 0 | |
| RandomModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 0 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 0 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 0 | |
| ENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=30%2026.06 | 0 | |
| RandomModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 0 |