General Language Evaluation on MMLU, BoolQ, OpenBookQA, RTE
70.4Average AccuracyMixtral-8x22B
Evaluation results
| Method | Links | |
|---|---|---|
| Mixtral-8x22BNumber of experts=82024.07 | 70.4 | |
| OriginalModel=Mixtral-8x22B, Scope=None2026.04 | 70.4 | |
| GRAPEModel=Mixtral-8x22B, Scope=Global, Number of experts pruned (e)=2e2026.04 | 68.2 | |
| Model-centricNumber of experts=62024.07 | 67.6 | |
| Data-centricNumber of experts=62024.07 | 67 | |
| DEKModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 67 | |
| EnumerateNumber of experts=62024.07 | 66.2 | |
| EnumerateModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 66.2 | |
| Mixtral-8x7BNumber of experts=8, Base Model=Mixtral-8x7B2024.07 | 66 | |
| Count-guidedNumber of experts=62024.07 | 63.3 | |
| Count-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 63.3 | |
| Model-centricNumber of experts=42024.07 | 62.6 | |
| GRAPEModel=Mixtral-8x22B, Scope=Global, Number of experts pruned (e)=4e2026.04 | 62.6 | |
| Data-centricNumber of experts=6, Base Model=Mixtral-8x7B2024.07 | 62.4 | |
| Model-centricNumber of experts=6, Base Model=Mixtral-8x7B2024.07 | 61.3 | |
| Data-centricNumber of experts=42024.07 | 61.1 | |
| DEKModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 61.1 | |
| EnumerateNumber of experts=6, Base Model=Mixtral-8x7B2024.07 | 59.9 | |
| EnumerateNumber of experts=42024.07 | 59.6 | |
| EnumerateModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 59.6 | |
| Router-guidedNumber of experts=6, Base Model=Mixtral-8x7B2024.07 | 57.8 | |
| Data-centricNumber of experts=4, Base Model=Mixtral-8x7B2024.07 | 54.5 | |
| Model-centricNumber of experts=4, Base Model=Mixtral-8x7B2024.07 | 54.2 | |
| Count-guidedNumber of experts=6, Base Model=Mixtral-8x7B2024.07 | 54.1 | |
| EnumerateNumber of experts=4, Base Model=Mixtral-8x7B2024.07 | 50.8 | |
| Count-guidedNumber of experts=42024.07 | 50 | |
| Count-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 50 | |
| OriginalModel=DeepSeek-MoE, Scope=None2026.04 | 50 | |
| GRAPEModel=DeepSeek-MoE, Scope=Global, Number of experts pruned (e)=2e2026.04 | 49.8 | |
| GRAPEModel=DeepSeek-MoE, Scope=Global, Number of experts pruned (e)=4e2026.04 | 49.5 | |
| Count-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 48.9 | |
| DEKModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 48.8 | |
| DEKModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 48.8 | |
| Count-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 47.5 | |
| Router-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 47.3 | |
| Count-guidedNumber of experts=4, Base Model=Mixtral-8x7B2024.07 | 45.7 | |
| Router-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 45.2 | |
| Router-guidedNumber of experts=62024.07 | 33.5 | |
| Router-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 33.5 | |
| Router-guidedNumber of experts=42024.07 | 31.6 | |
| Router-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 31.6 | |
| Router-guidedNumber of experts=4, Base Model=Mixtral-8x7B2024.07 | 29.1 |