Zero-shot Reasoning on Reasoning Tasks (PIQA, ARC-e, HS, WG)
80.41PIQA AccuracyLlama2-13B (W16A16)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Llama2-13B (W16A16)Backbone=Llama2-13B, #Bits=W16A162026.03 | 80.41 | 77.4 | 79.37 | 72.14 | 77.33 | — | |
| SliderQuantBackbone=Llama2-13B, #Bits=W4A162026.03 | 80.41 | 76.73 | 78.3 | 72.14 | 76.9 | — | |
| FP16Model=Llama-13B, #Bits=W16A162026.03 | 80.41 | 74.71 | 79.08 | 73.09 | 71.05 | 47.95 | |
| SliderQuant+Backbone=Llama2-13B, #Bits=W4A42026.03 | 79.96 | 77.27 | 77.96 | 71.98 | 76.79 | — | |
| LLM-MQBackbone=Llama2-13B, #Bits=W4A16 (0.5% in FP16)2026.03 | 79.49 | 58.5 | 76.31 | 69.3 | 70.9 | — | |
| LLM-MQBackbone=Llama2-13B, #Bits=W3.4A16 (0.5% in FP16)2026.03 | 79.49 | 58.12 | 74.77 | 69.61 | 70.5 | — | |
| FP16Model=Llama-7B, #Bits=W16A162026.03 | 79.43 | 73.15 | 76.16 | 70.24 | 68.81 | 45.05 | |
| SliderQuantModel=Llama-13B, #Bits=W3A162026.03 | 79.33 | 73.15 | 76.19 | 70.72 | 69.02 | 45.73 | |
| QUIKBackbone=Llama2-13B, #Bits=W4.9A4 (5% in FP16)2026.03 | 79.22 | 74.92 | 78.36 | 71.9 | 76.1 | — | |
| SliderQuantBackbone=Llama2-13B, #Bits=W3A162026.03 | 79.11 | 74.16 | 76.78 | 69.76 | 74.95 | — | |
| SpQRModel=Llama-13B, #Bits=W3.45A162026.03 | 78.73 | 73.27 | 58.22 | 68.9 | 64.37 | 42.75 | |
| SpQRModel=Llama-7B, #Bits=W3.45A162026.03 | 78.13 | 65.87 | 55.27 | 67.48 | 60.96 | 38.05 | |
| SliderQuantModel=Llama-7B, #Bits=W3A162026.03 | 77.42 | 69.7 | 71.74 | 67.96 | 65.44 | 40.36 | |
| SliderQuantBackbone=Llama2-13B, #Bits=W2.1A162026.03 | 77.21 | 62.58 | 71.05 | 67.51 | 69.59 | — | |
| LLM-MQBackbone=Llama2-13B, #Bits=W2.2A16 (0.5% in FP16)2026.03 | 76.77 | 55.26 | 70.83 | 67.09 | 67.49 | — | |
| SliderQuantBackbone=Llama2-13B, #Bits=W2A16 (0.5% in FP16)2026.03 | 76.13 | 67.51 | 69.25 | 66.12 | 69.75 | — | |
| LLM-MQBackbone=Llama2-13B, #Bits=W2A16 (0.5% in FP16)2026.03 | 75.84 | 54.29 | 68.32 | 65.51 | 65.99 | — | |
| SliderQuantBackbone=Llama2-13B, #Bits=W2A162026.03 | 73.56 | 67.47 | 64.75 | 63.22 | 67.25 | — |