Zero-shot Evaluation on Zero-shot Tasks (Task avg)
73.99Task Avg Scorebfloat16
Evaluation Results
| Method | Links | |
|---|---|---|
| bfloat16Bits=bfloat162026.04 | 73.99 | |
| OmniQuantBits=w8a162026.04 | 73.77 | |
| OmniQuantBits=w4a162026.04 | 73.62 | |
| Bit-by-BitBits=w8a162026.04 | 73.51 | |
| MatQuantBits=w8a162026.04 | 73.46 | |
| MatQuantBits=w4a162026.04 | 73.13 | |
| Bit-by-BitBits=w4a162026.04 | 72.21 | |
| MatQuantBits=w2a162026.04 | 65.99 | |
| Bit-by-BitBits=w2a162026.04 | 65.37 | |
| SAES-SVD + PARSEModel=Qwen2.5-7B, Compression Ratio=0.22026.05 | 61 | |
| OmniQuantBits=w2a162026.04 | 59.74 | |
| SAES-SVD + PARSEModel=LLaMA-30B, Compression Ratio=0.22026.05 | 59 | |
| FP ModelBackbone=Qwen-1.7B, Weight Precision=1-bit, Activation Precision=8-bit2026.05 | 58.1 | |
| FP ModelBackbone=Qwen-0.6B, Weight Precision=1-bit, Activation Precision=8-bit2026.05 | 58.1 | |
| SAES-SVDModel=Qwen2.5-7B, Compression Ratio=0.22026.05 | 58 | |
| SAES-SVDModel=LLaMA-30B, Compression Ratio=0.22026.05 | 57 | |
| SAES-SVD + PARSEModel=LLaMA-13B, Compression Ratio=0.22026.05 | 55 | |
| SAES-SVDModel=LLaMA-13B, Compression Ratio=0.22026.05 | 51 | |
| WINQBackbone=Qwen-1.7B, Weight Precision=2-bit, Activation Precision=8-bit2026.05 | 48.2 | |
| ParetoQBackbone=Qwen-1.7B, Weight Precision=2-bit, Activation Precision=8-bit2026.05 | 47.8 | |
| WINQBackbone=Qwen-0.6B, Weight Precision=2-bit, Activation Precision=8-bit2026.05 | 43.9 | |
| ParetoQBackbone=Qwen-0.6B, Weight Precision=2-bit, Activation Precision=8-bit2026.05 | 43.3 | |
| ParetoQBackbone=Qwen-1.7B, Weight Precision=1-bit, Activation Precision=8-bit2026.05 | 42.2 | |
| WINQBackbone=Qwen-1.7B, Weight Precision=1-bit, Activation Precision=8-bit2026.05 | 42.2 | |
| WINQBackbone=Qwen-0.6B, Weight Precision=1-bit, Activation Precision=8-bit2026.05 | 41.4 | |
| ParetoQBackbone=Qwen-0.6B, Weight Precision=1-bit, Activation Precision=8-bit2026.05 | 41.2 |