List your company
ResearchFormerly WizWand

Zero-shot Evaluation on Zero-shot Tasks (Task avg)

73.99Task Avg Scorebfloat16
39.888448.741757.59566.4483Apr 9, 2026Apr 15, 2026Apr 21, 2026Apr 28, 2026May 4, 2026May 10, 2026May 17, 2026
Updated 5mo ago

Evaluation results

MethodLinks
bfloat16Bits=bfloat162026.0473.99
OmniQuantBits=w8a162026.0473.77
OmniQuantBits=w4a162026.0473.62
Bit-by-BitBits=w8a162026.0473.51
MatQuantBits=w8a162026.0473.46
MatQuantBits=w4a162026.0473.13
Bit-by-BitBits=w4a162026.0472.21
MatQuantBits=w2a162026.0465.99
Bit-by-BitBits=w2a162026.0465.37
SAES-SVD + PARSEModel=Qwen2.5-7B, Compression Ratio=0.22026.0561
OmniQuantBits=w2a162026.0459.74
SAES-SVD + PARSEModel=LLaMA-30B, Compression Ratio=0.22026.0559
FP ModelBackbone=Qwen-1.7B, Weight Precision=1-bit, Activation Precision=8-bit2026.0558.1
FP ModelBackbone=Qwen-0.6B, Weight Precision=1-bit, Activation Precision=8-bit2026.0558.1
SAES-SVDModel=Qwen2.5-7B, Compression Ratio=0.22026.0558
SAES-SVDModel=LLaMA-30B, Compression Ratio=0.22026.0557
SAES-SVD + PARSEModel=LLaMA-13B, Compression Ratio=0.22026.0555
SAES-SVDModel=LLaMA-13B, Compression Ratio=0.22026.0551
WINQBackbone=Qwen-1.7B, Weight Precision=2-bit, Activation Precision=8-bit2026.0548.2
ParetoQBackbone=Qwen-1.7B, Weight Precision=2-bit, Activation Precision=8-bit2026.0547.8
WINQBackbone=Qwen-0.6B, Weight Precision=2-bit, Activation Precision=8-bit2026.0543.9
ParetoQBackbone=Qwen-0.6B, Weight Precision=2-bit, Activation Precision=8-bit2026.0543.3
ParetoQBackbone=Qwen-1.7B, Weight Precision=1-bit, Activation Precision=8-bit2026.0542.2
WINQBackbone=Qwen-1.7B, Weight Precision=1-bit, Activation Precision=8-bit2026.0542.2
WINQBackbone=Qwen-0.6B, Weight Precision=1-bit, Activation Precision=8-bit2026.0541.4
ParetoQBackbone=Qwen-0.6B, Weight Precision=1-bit, Activation Precision=8-bit2026.0541.2