Aggregate Evaluation on GSM8K, MBPP, HumanEval, ARC-E, and ARC-C
61.16Average ScoreFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullModel=Qwen1.5MoE-A2.7B, E↓=100%, A↓=100%2026.06 | 61.16 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=61.38%2026.06 | 59.06 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=60%2026.06 | 55.88 | |
| FullModel=DeepSeek-V2-Lite, E↓=100%, A↓=100%2026.06 | 52.98 | |
| TENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=63.76%2026.06 | 51.98 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 48.02 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 46.18 | |
| ENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=60%2026.06 | 43.8 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 43.14 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=34.51%2026.06 | 43 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=30%2026.06 | 41.9 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 41.18 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 39.78 | |
| TENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=36.38%2026.06 | 39.72 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 39.06 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 32.62 | |
| ENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=30%2026.06 | 24.16 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 21.14 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 20.68 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 19.66 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 15.26 | |
| RandomModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 13.56 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 11.28 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 10.64 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 10.04 | |
| RandomModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 9.96 |