List your company
ResearchFormerly WizWand

Average Zero-shot Performance on 8-Task Aggregate

76.31Accuracy (Zero-Shot Aggregate)Full-precision
34.938845.679456.4267.1606Apr 7, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Full-precisionModel=Mixtral 8x22B, Avg. bits/exp.=16 (FP), Memory (GB)=281.2, Evaluation protocol=zero-shot2026.0476.31
Full-precisionModel=Mixtral 8x7B, Avg. bits/exp.=16 (FP), Memory (GB)=96.8, Evaluation protocol=zero-shot2026.0472.72
UniformModel=Mixtral 8x7B, Avg. bits/exp.=3, Memory (GB)=19.3, Evaluation protocol=zero-shot2026.0470.85
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.75, Memory (GB)=17.7, Evaluation protocol=zero-shot2026.0470.01
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.75, Memory (GB)=17.7, Evaluation protocol=zero-shot2026.0469.85
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.625, Memory (GB)=16.9, Evaluation protocol=zero-shot2026.0469.04
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.625, Memory (GB)=16.9, Evaluation protocol=zero-shot2026.0468.67
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=16.1, Evaluation protocol=zero-shot2026.0468.38
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=16.1, Evaluation protocol=zero-shot2026.0467.53
HessianModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=17.0, Evaluation protocol=zero-shot2026.0467.18
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.375, Memory (GB)=15.3, Evaluation protocol=zero-shot2026.0466.72
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.375, Memory (GB)=15.3, Evaluation protocol=zero-shot2026.0465.93
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.25, Memory (GB)=14.5, Evaluation protocol=zero-shot2026.0465.79
UniformModel=Mixtral 8x22B, Avg. bits/exp.=3, Memory (GB)=57.5, Evaluation protocol=zero-shot2026.0465.48
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.25, Memory (GB)=14.5, Evaluation protocol=zero-shot2026.0465.16
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.125, Memory (GB)=13.8, Evaluation protocol=zero-shot2026.0464.26
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.125, Memory (GB)=13.8, Evaluation protocol=zero-shot2026.0463.79
HessianModel=Mixtral 8x7B, Avg. bits/exp.=2.25, Memory (GB)=15.3, Evaluation protocol=zero-shot2026.0463.47
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.0, Memory (GB)=13.1, Evaluation protocol=zero-shot2026.0462.83
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.0, Memory (GB)=13.1, Evaluation protocol=zero-shot2026.0462.56
PMQModel=Mixtral 8x22B, Avg. bits/exp.=2.5, Memory (GB)=46.7, Evaluation protocol=zero-shot2026.0460.69
PMQModel=Mixtral 8x7B, Avg. bits/exp.=1.75, Memory (GB)=11.7, Evaluation protocol=zero-shot2026.0460.53
Router norm + Max varModel=Mixtral 8x22B, Avg. bits/exp.=2.5, Memory (GB)=46.7, Evaluation protocol=zero-shot2026.0460.1
Router norm + Max varModel=Mixtral 8x22B, Avg. bits/exp.=2.25, Memory (GB)=43.0, Evaluation protocol=zero-shot2026.0459.17
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=1.75, Memory (GB)=11.7, Evaluation protocol=zero-shot2026.0458.95
HessianModel=Mixtral 8x7B, Avg. bits/exp.=2.0, Memory (GB)=13.6, Evaluation protocol=zero-shot2026.0458.85
UniformModel=Mixtral 8x7B, Avg. bits/exp.=2, Memory (GB)=13.1, Evaluation protocol=zero-shot2026.0458.73
PMQModel=Mixtral 8x22B, Avg. bits/exp.=2.25, Memory (GB)=43.0, Evaluation protocol=zero-shot2026.0455.35
Router norm + Max varModel=Mixtral 8x22B, Avg. bits/exp.=2.0, Memory (GB)=38.6, Evaluation protocol=zero-shot2026.0455.34
PMQModel=Mixtral 8x22B, Avg. bits/exp.=2.0, Memory (GB)=38.6, Evaluation protocol=zero-shot2026.0454.8
Router norm + Max varModel=Mixtral 8x22B, Avg. bits/exp.=1.75, Memory (GB)=35.2, Evaluation protocol=zero-shot2026.0451.44
BSPModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=17.0, Evaluation protocol=zero-shot2026.0449.07
PMQModel=Mixtral 8x22B, Avg. bits/exp.=1.75, Memory (GB)=35.2, Evaluation protocol=zero-shot2026.0447.87
Slim-LLMModel=Mixtral 8x7B, Avg. bits/exp.=2.0, Memory (GB)=13.6, Evaluation protocol=zero-shot2026.0444.49
UniformModel=Mixtral 8x22B, Avg. bits/exp.=2, Memory (GB)=38.6, Evaluation protocol=zero-shot2026.0436.53