Zero-shot Evaluation on MMLU, GSM, and M500 (test)
68.3Avg AccuracyGeneric TB-Coverage
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Generic TB-CoverageModel=DeepSeek-MoE-16B-Base, Retain Ratio=75%2026.07 | 68.3 | 30.9 | 0 | 0.6 | |
| Paper ExpertSparsityModel=DeepSeek-MoE-16B-Base, Retain Ratio=75%2026.07 | 67.8 | 31.4 | 0 | 0.4 | |
| Generic TB-CoverageModel=Qwen1.5-MoE-A2.7B, Retain Ratio=75%2026.07 | 66.2 | 52.8 | 0.2 | 0.2 | |
| Paper ExpertSparsityModel=Qwen1.5-MoE-A2.7B, Retain Ratio=75%2026.07 | 65.5 | 54.8 | 0 | 0.2 | |
| Original REAPModel=DeepSeek-MoE-16B-Base, Retain Ratio=75%2026.07 | 64.9 | 34 | 0 | 0 | |
| RandomModel=Qwen1.5-MoE-A2.7B, Retain Ratio=75%2026.07 | 64.5 | 51.6 | 0 | 0.3 | |
| Original REAPModel=Qwen1.5-MoE-A2.7B, Retain Ratio=75%2026.07 | 62.9 | 51.9 | 0 | 0.2 | |
| Generic TB-CoverageModel=DeepSeek-MoE-16B-Base, Retain Ratio=50%2026.07 | 60.6 | 24.2 | 0 | 0.6 | |
| Generic TB-CoverageModel=Qwen1.5-MoE-A2.7B, Retain Ratio=50%2026.07 | 59.8 | 36.9 | 0 | 0.4 | |
| Paper ExpertSparsityModel=DeepSeek-MoE-16B-Base, Retain Ratio=50%2026.07 | 57.1 | 26.4 | 0 | 1 | |
| RandomModel=DeepSeek-MoE-16B-Base, Retain Ratio=75%2026.07 | 56.8 | 26 | 0 | 0.6 | |
| Original REAPModel=Qwen1.5-MoE-A2.7B, Retain Ratio=50%2026.07 | 55.2 | 40.6 | 0 | 0.2 | |
| Original REAPModel=DeepSeek-MoE-16B-Base, Retain Ratio=50%2026.07 | 54.2 | 28.7 | 0 | 0.8 | |
| RandomModel=Qwen1.5-MoE-A2.7B, Retain Ratio=50%2026.07 | 53.5 | 31.5 | 0 | 0.2 | |
| Paper ExpertSparsityModel=Qwen1.5-MoE-A2.7B, Retain Ratio=50%2026.07 | 51.9 | 32.5 | 0 | 0 | |
| RandomModel=DeepSeek-MoE-16B-Base, Retain Ratio=50%2026.07 | 48.8 | 24.4 | 0 | 0.5 | |
| Generic TB-CoverageModel=Qwen1.5-MoE-A2.7B, Retain Ratio=25%2026.07 | 44.8 | 23.4 | 0 | 0 | |
| Generic TB-CoverageModel=DeepSeek-MoE-16B-Base, Retain Ratio=25%2026.07 | 44.8 | 23.2 | 0 | 0 | |
| Original REAPModel=Qwen1.5-MoE-A2.7B, Retain Ratio=25%2026.07 | 43.8 | 24.7 | 0 | 0 | |
| RandomModel=Qwen1.5-MoE-A2.7B, Retain Ratio=25%2026.07 | 41.7 | 24 | — | — | |
| Paper ExpertSparsityModel=Qwen1.5-MoE-A2.7B, Retain Ratio=25%2026.07 | 40.7 | 23.3 | 0 | 0 | |
| Paper ExpertSparsityModel=DeepSeek-MoE-16B-Base, Retain Ratio=25%2026.07 | 40.4 | 24 | 0 | 0 | |
| RandomModel=DeepSeek-MoE-16B-Base, Retain Ratio=25%2026.07 | 40.3 | 24.3 | — | — | |
| Original REAPModel=DeepSeek-MoE-16B-Base, Retain Ratio=25%2026.07 | 39.5 | 23.8 | 0 | 0.6 |