Code Generation on MBPP (MBPP)
47.6MBPP AccuracyFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullModel=Qwen1.5MoE-A2.7B, E↓=100%, A↓=100%2026.06 | 47.6 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=61.38%2026.06 | 45.7 | |
| TENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=63.76%2026.06 | 45.7 | |
| FullModel=DeepSeek-V2-Lite, E↓=100%, A↓=100%2026.06 | 43.2 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 42.1 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=60%2026.06 | 40.6 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 35.4 | |
| ENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=60%2026.06 | 33.9 | |
| TENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=36.38%2026.06 | 33.1 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 30.7 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=34.51%2026.06 | 25.3 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=30%2026.06 | 25.1 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 24.4 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 18.9 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 14.6 | |
| ENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=30%2026.06 | 9.1 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 4.3 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 1.2 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 0.4 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 0.4 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 0 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 0 | |
| RandomModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 0 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 0 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 0 | |
| RandomModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 0 |