Average Zero-shot Performance on 8-Task Aggregate
76.31Accuracy (Zero-Shot Aggregate)Full-precision
Evaluation Results
| Method | Links | |
|---|---|---|
| Full-precisionModel=Mixtral 8x22B, Avg. bits/exp.=16 (FP), Memory (GB)=281.2, Evaluation protocol=zero-shot2026.04 | 76.31 | |
| Full-precisionModel=Mixtral 8x7B, Avg. bits/exp.=16 (FP), Memory (GB)=96.8, Evaluation protocol=zero-shot2026.04 | 72.72 | |
| UniformModel=Mixtral 8x7B, Avg. bits/exp.=3, Memory (GB)=19.3, Evaluation protocol=zero-shot2026.04 | 70.85 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.75, Memory (GB)=17.7, Evaluation protocol=zero-shot2026.04 | 70.01 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.75, Memory (GB)=17.7, Evaluation protocol=zero-shot2026.04 | 69.85 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.625, Memory (GB)=16.9, Evaluation protocol=zero-shot2026.04 | 69.04 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.625, Memory (GB)=16.9, Evaluation protocol=zero-shot2026.04 | 68.67 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=16.1, Evaluation protocol=zero-shot2026.04 | 68.38 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=16.1, Evaluation protocol=zero-shot2026.04 | 67.53 | |
| HessianModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=17.0, Evaluation protocol=zero-shot2026.04 | 67.18 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.375, Memory (GB)=15.3, Evaluation protocol=zero-shot2026.04 | 66.72 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.375, Memory (GB)=15.3, Evaluation protocol=zero-shot2026.04 | 65.93 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.25, Memory (GB)=14.5, Evaluation protocol=zero-shot2026.04 | 65.79 | |
| UniformModel=Mixtral 8x22B, Avg. bits/exp.=3, Memory (GB)=57.5, Evaluation protocol=zero-shot2026.04 | 65.48 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.25, Memory (GB)=14.5, Evaluation protocol=zero-shot2026.04 | 65.16 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.125, Memory (GB)=13.8, Evaluation protocol=zero-shot2026.04 | 64.26 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.125, Memory (GB)=13.8, Evaluation protocol=zero-shot2026.04 | 63.79 | |
| HessianModel=Mixtral 8x7B, Avg. bits/exp.=2.25, Memory (GB)=15.3, Evaluation protocol=zero-shot2026.04 | 63.47 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.0, Memory (GB)=13.1, Evaluation protocol=zero-shot2026.04 | 62.83 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.0, Memory (GB)=13.1, Evaluation protocol=zero-shot2026.04 | 62.56 | |
| PMQModel=Mixtral 8x22B, Avg. bits/exp.=2.5, Memory (GB)=46.7, Evaluation protocol=zero-shot2026.04 | 60.69 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=1.75, Memory (GB)=11.7, Evaluation protocol=zero-shot2026.04 | 60.53 | |
| Router norm + Max varModel=Mixtral 8x22B, Avg. bits/exp.=2.5, Memory (GB)=46.7, Evaluation protocol=zero-shot2026.04 | 60.1 | |
| Router norm + Max varModel=Mixtral 8x22B, Avg. bits/exp.=2.25, Memory (GB)=43.0, Evaluation protocol=zero-shot2026.04 | 59.17 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=1.75, Memory (GB)=11.7, Evaluation protocol=zero-shot2026.04 | 58.95 | |
| HessianModel=Mixtral 8x7B, Avg. bits/exp.=2.0, Memory (GB)=13.6, Evaluation protocol=zero-shot2026.04 | 58.85 | |
| UniformModel=Mixtral 8x7B, Avg. bits/exp.=2, Memory (GB)=13.1, Evaluation protocol=zero-shot2026.04 | 58.73 | |
| PMQModel=Mixtral 8x22B, Avg. bits/exp.=2.25, Memory (GB)=43.0, Evaluation protocol=zero-shot2026.04 | 55.35 | |
| Router norm + Max varModel=Mixtral 8x22B, Avg. bits/exp.=2.0, Memory (GB)=38.6, Evaluation protocol=zero-shot2026.04 | 55.34 | |
| PMQModel=Mixtral 8x22B, Avg. bits/exp.=2.0, Memory (GB)=38.6, Evaluation protocol=zero-shot2026.04 | 54.8 | |
| Router norm + Max varModel=Mixtral 8x22B, Avg. bits/exp.=1.75, Memory (GB)=35.2, Evaluation protocol=zero-shot2026.04 | 51.44 | |
| BSPModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=17.0, Evaluation protocol=zero-shot2026.04 | 49.07 | |
| PMQModel=Mixtral 8x22B, Avg. bits/exp.=1.75, Memory (GB)=35.2, Evaluation protocol=zero-shot2026.04 | 47.87 | |
| Slim-LLMModel=Mixtral 8x7B, Avg. bits/exp.=2.0, Memory (GB)=13.6, Evaluation protocol=zero-shot2026.04 | 44.49 | |
| UniformModel=Mixtral 8x22B, Avg. bits/exp.=2, Memory (GB)=38.6, Evaluation protocol=zero-shot2026.04 | 36.53 |