Zero-shot Commonsense Reasoning Suite (PIQA, ARC, HS, WG, BoolQ, MMLU)
82.1PIQA Accuracy (Zero-shot)W16A16 Baseline
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| W16A16 BaselineModel=Qwen2.5-14B, #Bits=W16A16, Evaluation Protocol=Zero-shot2026.03 | 82.1 | 79.59 | 58.87 | 82.95 | 75.61 | 85.26 | 77.58 | 77.42 | |
| W16A16 BaselineModel=Llama2-13B, #Bits=W16A16, Evaluation Protocol=Zero-shot2026.03 | 80.41 | 77.4 | 49.15 | 79.37 | 72.14 | 80.55 | 52.77 | 70.26 | |
| SliderQuantModel=Llama2-13B, #Bits=W4A4, Evaluation Protocol=Zero-shot2026.03 | 71.65 | 62.88 | 37.8 | 66.02 | 60.77 | 71.22 | 27.04 | 56.77 | |
| SliderQuantModel=Qwen2.5-14B, #Bits=W4A4, Evaluation Protocol=Zero-shot2026.03 | 71.16 | 66.96 | 40.96 | 63.38 | 62.51 | 64.25 | 43.5 | 58.96 | |
| CBQModel=Llama2-13B, #Bits=W4A4, Evaluation Protocol=Zero-shot2026.03 | 71 | 61.57 | 35.84 | 65.15 | 57.93 | 66.39 | 24.78 | 54.67 | |
| OmniQuantModel=Llama2-13B, #Bits=W4A4, Evaluation Protocol=Zero-shot2026.03 | 69.21 | 57.37 | 34.56 | 61.95 | 56.91 | 65.44 | 23.56 | 52.71 | |
| CBQModel=Qwen2.5-14B, #Bits=W4A4, Evaluation Protocol=Zero-shot2026.03 | 67.52 | 60.86 | 36.18 | 60.12 | 58.09 | 60.15 | 31.61 | 53.5 | |
| SmoothQuantModel=Llama2-13B, #Bits=W4A4, Evaluation Protocol=Zero-shot2026.03 | 61.1 | 44.87 | 27.47 | 41.03 | 50.67 | 58.5 | 21.14 | 43.54 | |
| OmniQuantModel=Qwen2.5-14B, #Bits=W4A4, Evaluation Protocol=Zero-shot2026.03 | 59.45 | 48.56 | 30.16 | 61.42 | 54.23 | 58.34 | 27.68 | 48.55 | |
| SmoothQuantModel=Qwen2.5-14B, #Bits=W4A4, Evaluation Protocol=Zero-shot2026.03 | 54.57 | 35.14 | 24.66 | 35.29 | 51.46 | 56.45 | 24.9 | 40.35 |