Zero-shot Language Understanding on BoolQ, ARC-e, ARC-c, WinoGrande, and HellaSwag
83.08ARC-e AccuracyMoEITS
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MoEITSBackbone=Mixtral 8 x 7B, P ↓=43%2026.04 | 83.08 | 87.33 | 55.89 | 80.98 | 82.83 | 78.02 | |
| MoPBackbone=Mixtral 8 x 7B, P ↓=25%2026.04 | 82.37 | 86.51 | 75.43 | 55.8 | 74.28 | 74.88 | |
| MoE-PrunerBackbone=Mixtral 8 x 7B, P ↓=0%, S=50%2026.04 | 81.9 | 86 | 53.3 | 75.5 | 62.3 | 71.8 | |
| C-GNN-PRUNEBackbone=Mixtral 8 x 7B, P ↓=50%2026.04 | 80.03 | 83.73 | — | 72.37 | 56.16 | 73.1 | |
| STUNBackbone=Mixtral 8 x 7B, P ↓=0%, S=65%2026.04 | 80 | — | 51.5 | — | 59.9 | 63.8 | |
| Lu et al.Backbone=Mixtral 8 x 7B, P ↓=40%2026.04 | 79.9 | 84.8 | 53.9 | 73.8 | 60.1 | 70.5 | |
| MoE-I^2Backbone=Mixtral 8 x 7B, P ↓=49%2026.04 | 78.2 | 82.6 | 52.2 | 71.5 | 61.1 | 69.1 | |
| MC-MoEBackbone=Mixtral 8 x 7B2026.04 | 73.1 | 80.6 | 48.4 | 71.3 | 74.9 | 69.6 |