Zero-shot Evaluation on ARC-c, ARC-e, HellaSwag, PIQA, and WinoGrande
59.3ARC-c AccuracyOriginal
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| OriginalModel=Mixtral-8×7B, Bits=162026.06 | 59.3 | 83.67 | 84.03 | 83.57 | 76.24 | 77.36 | |
| AlphaQModel=Mixtral-8×7B, Bits=3.52026.06 | 55.8 | 83.38 | 74.67 | 81.51 | 76.1 | 74.29 | |
| OriginalModel=Qwen3-30B-A3B, Bits=162026.06 | 55.8 | 79.29 | 77.75 | 80.25 | 71.11 | 72.84 | |
| UniformModel=Mixtral-8×7B, Bits=3.52026.06 | 54.95 | 83.04 | 73.58 | 81.28 | 76.19 | 73.81 | |
| PMQModel=Mixtral-8×7B, Bits=3.52026.06 | 53.92 | 82.49 | 72.9 | 81.21 | 76.09 | 73.32 | |
| AlphaQModel=Mixtral-8×7B, Bits=2.52026.06 | 51.59 | 78.87 | 69.24 | 80.84 | 73.24 | 70.76 | |
| PMQModel=Mixtral-8×7B, Bits=2.52026.06 | 51.28 | 77.1 | 69.23 | 80.36 | 73.95 | 70.38 | |
| HessianModel=Mixtral-8×7B, Bits=2.52026.06 | 51.2 | 76.38 | 67.05 | 80.21 | 72.97 | 69.56 | |
| AlphaQModel=Qwen3-30B-A3B, Bits=2.52026.06 | 50.35 | 75.87 | 74.49 | 78.34 | 68.71 | 69.56 | |
| FP16#W=16, Size%=100.02026.04 | 50.3 | 80.1 | 60.2 | 79.5 | 73.7 | 68.8 | |
| UniformModel=Mixtral-8×7B, Bits=2.52026.06 | 50.26 | 77.4 | 58.84 | 79.38 | 73.4 | 67.86 | |
| UniformModel=Qwen3-30B-A3B, Bits=2.52026.06 | 50.07 | 75.93 | 71.72 | 78.07 | 68.01 | 68.76 | |
| FASQ 2-512Calib Free?=true, #W=4.5, Size%=41.92026.04 | 50 | 78.5 | 59.5 | 78.7 | 73.4 | 68 | |
| FASQ 2-1024Calib Free?=true, #W=5.0, Size%=49.02026.04 | 49.7 | 79 | 59.9 | 78.9 | 73.3 | 68.2 | |
| SmoothQuant (W8A8)Calib Free?=false, #W=W8A8, Size%=56.62026.04 | 49 | 79.6 | 60 | 79.4 | 73.2 | 68.2 | |
| OriginalModel=DeepSeekV2-Lite, Bits=162026.06 | 48.98 | 76.47 | 78.01 | 80.08 | 71.51 | 71.01 | |
| AWQCalib Free?=false, #W=4, Size%=35.7, Group Size=1282026.04 | 48.5 | 79.3 | 59.1 | 78.6 | 73.1 | 67.7 | |
| NeUQIModel=Qwen 2.5, Size=72B, Bits=22025.05 | 48.04 | 78.11 | 56.18 | 78.56 | 75.14 | 67.21 | |
| FASQ 2-256Calib Free?=true, #W=4.0, Size%=37.02026.04 | 48 | 79.5 | 58.5 | 79.4 | 73.4 | 67.8 | |
| GPTQCalib Free?=false, #W=4, Size%=35.7, Group Size=1282026.04 | 47.7 | 78.8 | 59 | 78.4 | 72.6 | 67.3 | |
| QuIPCalib Free?=false, #W=4, Size%=34.82026.04 | 47.4 | 78.2 | 58.6 | 78.2 | 73.2 | 67.1 | |
| AlphaQModel=DeepSeekV2-Lite, Bits=3.52026.06 | 46.51 | 75.51 | 77.21 | 79.89 | 71.21 | 70.07 | |
| FASQ 2-128Calib Free?=true, #W=3.5, Size%=33.22026.04 | 46.3 | 77.1 | 56.4 | 78.2 | 72.2 | 66 | |
| PMQModel=Qwen3-30B-A3B, Bits=2.52026.06 | 45.99 | 68.43 | 72.13 | 76.61 | 66.22 | 65.88 | |
| PMQModel=DeepSeekV2-Lite, Bits=3.52026.06 | 45.83 | 75.26 | 76.97 | 79.63 | 71.04 | 69.75 | |
| SmoothQuant (W6A6)Calib Free?=false, #W=W6A6, Size%=56.62026.04 | 45 | 75.5 | 56.9 | 76.8 | 69 | 64.6 | |
| NeUQIModel=LLaMA 2, Size=70B, Bits=22025.05 | 44.71 | 75.97 | 53.62 | 77.64 | 73.72 | 65.13 | |
| UniformModel=DeepSeekV2-Lite, Bits=3.52026.06 | 44.57 | 74.56 | 76.27 | 79.21 | 70.83 | 69.09 | |
| AlphaQModel=Qwen1.5-MoE, Bits=3.52026.06 | 44.33 | 69.15 | 77.7 | 80.46 | 68.55 | 68.04 | |
| OriginalModel=Qwen1.5-MoE, Bits=162026.06 | 44.11 | 69.23 | 77.21 | 80.52 | 68.59 | 67.93 | |
| PMQModel=Qwen1.5-MoE, Bits=3.52026.06 | 43.81 | 69.02 | 77.13 | 80.31 | 68.5 | 67.75 | |
| AWQCalib Free?=false, #W=3, Size%=30.2, Group Size=1282026.04 | 43.2 | 74 | 55.1 | 77.7 | 72.1 | 64.4 | |
| AlphaQModel=Qwen1.5-MoE, Bits=2.52026.06 | 42.72 | 66.97 | 71.81 | 78.21 | 66.12 | 65.17 | |
| NeUQIModel=Qwen 2.5, Size=32B, Bits=22025.05 | 41.55 | 71.63 | 53.38 | 76.01 | 72.77 | 63.07 | |
| UniformModel=Qwen1.5-MoE, Bits=3.52026.06 | 41.51 | 66.81 | 74.52 | 79.31 | 67.54 | 65.94 | |
| QuIPCalib Free?=false, #W=3, Size%=29.42026.04 | 41 | 72.9 | 55.4 | 76.8 | 72.5 | 63.7 | |
| MagRModel=LLaMA 2, Size=70B, Bits=2, extra coordinate descent iterations=false2025.05 | 39.59 | 71.21 | 52.81 | 76.33 | 69.06 | 61.8 | |
| RTNCalib Free?=true, #W=4, Size%=35.7, Group Size=1282026.04 | 39.4 | 68.2 | 56 | 75 | 69 | 61.5 | |
| AlphaQModel=DeepSeekV2-Lite, Bits=2.52026.06 | 39.22 | 73.77 | 73.38 | 78.02 | 69.3 | 66.74 | |
| NeUQIModel=Qwen 2.5, Size=14B, Bits=22025.05 | 37.88 | 69.32 | 46.86 | 72.31 | 67.4 | 58.76 | |
| GPTQCalib Free?=false, #W=3, Size%=30.2, Group Size=1282026.04 | 37.7 | 70.5 | 54.3 | 74.9 | 71.1 | 61.7 | |
| AFGModel=Mixtral-8×7B, Bits=2.52026.06 | 37.57 | 69.2 | 67.88 | 74.32 | 63.8 | 62.55 | |
| AFGModel=DeepSeekV2-Lite, Bits=2.52026.06 | 37.57 | 69.2 | 67.88 | 74.32 | 63.8 | 62.55 | |
| PMQModel=DeepSeekV2-Lite, Bits=2.52026.06 | 37.37 | 71.68 | 72.76 | 77.86 | 69.06 | 65.75 | |
| PMQModel=Qwen1.5-MoE, Bits=2.52026.06 | 36.26 | 60.56 | 70.69 | 77.58 | 65.82 | 62.18 | |
| UniformModel=DeepSeekV2-Lite, Bits=2.52026.06 | 35.84 | 71.34 | 72.44 | 76.66 | 69.46 | 65.15 | |
| MagRModel=Qwen 2.5, Size=72B, Bits=2, extra coordinate descent iterations=true2025.05 | 31.91 | 59.64 | 51.13 | 72.74 | 59.59 | 55 | |
| AFGModel=Qwen1.5-MoE, Bits=2.52026.06 | 31.2 | 61.07 | 72.77 | 77.53 | 62.12 | 60.94 | |
| UniformModel=Qwen1.5-MoE, Bits=2.52026.06 | 29.44 | 49.07 | 53.98 | 71.49 | 55.96 | 51.99 | |
| BSPModel=Mixtral-8×7B, Bits=2.52026.06 | 28.38 | 54.97 | 55.61 | 68.23 | 62.19 | 53.88 | |
| NeUQIModel=LLaMA 2, Size=13B, Bits=22025.05 | 26.19 | 55.18 | 37.37 | 65.94 | 59.12 | 48.76 | |
| MagRModel=Qwen 2.5, Size=32B, Bits=2, extra coordinate descent iterations=true2025.05 | 25.94 | 48.91 | 45.25 | 68.06 | 56.51 | 48.93 | |
| NeUQIModel=LLaMA 3, Size=70B, Bits=22025.05 | 25.68 | 53.24 | 36.98 | 61.04 | 55.49 | 46.49 | |
| NeUQIModel=LLaMA 3, Size=8B, Bits=22025.05 | 24.83 | 52.1 | 37.5 | 63 | 59.04 | 47.3 | |
| NeUQIModel=LLaMA 2, Size=7B, Bits=22025.05 | 23.98 | 51.73 | 36.04 | 65.89 | 58.56 | 47.24 | |
| MagRModel=LLaMA 2, Size=13B, Bits=2, extra coordinate descent iterations=false2025.05 | 23.21 | 26.68 | 36.7 | 52.99 | 54.14 | 38.74 | |
| GPTAQModel=Qwen 2.5, Size=7B, Bits=22025.05 | 23.21 | 25.34 | 26.49 | 52.23 | 46.96 | 34.85 | |
| MagRModel=Qwen 2.5, Size=14B, Bits=2, extra coordinate descent iterations=true2025.05 | 23.21 | 44.74 | 38.06 | 63.44 | 52.88 | 44.47 | |
| MagRModel=Qwen 2.5, Size=7B, Bits=2, extra coordinate descent iterations=true2025.05 | 23.12 | 42.09 | 37.76 | 60.99 | 51.3 | 43.05 | |
| GPTQModel=Qwen 2.5, Size=14B, Bits=22025.05 | 23.12 | 25.42 | 25.9 | 50.92 | 51.93 | 35.46 | |
| GPTQModel=Qwen 2.5, Size=72B, Bits=22025.05 | 22.95 | 25 | 25.46 | 51.31 | 48.7 | 34.68 | |
| GPTAQModel=LLaMA 3, Size=70B, Bits=22025.05 | 22.7 | 24.87 | 25.78 | 53.21 | 51.22 | 35.56 | |
| GPTAQModel=Qwen 2.5, Size=14B, Bits=22025.05 | 22.35 | 25.63 | 25.6 | 51.31 | 49.8 | 34.94 | |
| NeUQIModel=Qwen 2.5, Size=7B, Bits=22025.05 | 22.1 | 47.85 | 39.56 | 60.99 | 58.48 | 45.8 | |
| GPTQModel=LLaMA 2, Size=7B, Bits=22025.05 | 21.93 | 25.63 | 25.89 | 52.23 | 49.72 | 35.08 | |
| GPTAQModel=LLaMA 2, Size=7B, Bits=22025.05 | 21.76 | 26.89 | 25.74 | 53.32 | 47.12 | 34.97 | |
| GPTQModel=LLaMA 2, Size=13B, Bits=22025.05 | 21.76 | 26.52 | 25.85 | 52.56 | 51.3 | 35.6 | |
| GPTAQModel=LLaMA 2, Size=70B, Bits=22025.05 | 21.59 | 28.91 | 28.98 | 54.08 | 51.46 | 37 | |
| MagRModel=LLaMA 3, Size=70B, Bits=2, extra coordinate descent iterations=true2025.05 | 21.59 | 25.59 | 25.51 | 52.99 | 48.46 | 34.83 | |
| GPTAQModel=Qwen 2.5, Size=72B, Bits=22025.05 | 21.59 | 23.4 | 27.87 | 51.74 | 49.09 | 34.74 | |
| MagRModel=LLaMA 2, Size=7B, Bits=2, extra coordinate descent iterations=false2025.05 | 21.42 | 34.97 | 32.29 | 58.27 | 50.75 | 39.54 | |
| GPTQModel=Qwen 2.5, Size=7B, Bits=22025.05 | 21.42 | 26.52 | 25.7 | 50.92 | 48.93 | 34.7 | |
| GPTQModel=Qwen 2.5, Size=32B, Bits=22025.05 | 21.42 | 26.09 | 25.11 | 51.2 | 49.41 | 34.65 | |
| GPTAQModel=LLaMA 3, Size=8B, Bits=22025.05 | 20.99 | 25.29 | 25.88 | 52.99 | 48.46 | 34.72 | |
| GPTQModel=LLaMA 3, Size=8B, Bits=22025.05 | 20.9 | 23.91 | 26.04 | 53.48 | 48.86 | 34.64 | |
| GPTAQModel=Qwen 2.5, Size=32B, Bits=22025.05 | 20.82 | 23.86 | 27.08 | 52.23 | 51.22 | 35.04 | |
| GPTQModel=LLaMA 3, Size=70B, Bits=22025.05 | 20.73 | 25.76 | 25.76 | 52.45 | 48.46 | 34.63 | |
| GPTAQModel=LLaMA 2, Size=13B, Bits=22025.05 | 20.56 | 26.89 | 26.99 | 53.43 | 51.85 | 35.95 | |
| SmoothQuant (W4A4)Calib Free?=false, #W=W4A4, Size%=34.82026.04 | 20 | 26.3 | 26.4 | 54.6 | 50.3 | 35.5 | |
| RTNCalib Free?=true, #W=3, Size%=30.2, Group Size=1282026.04 | 20 | 31.1 | 27.5 | 56.2 | 53.1 | 37.6 | |
| GPTQModel=LLaMA 2, Size=70B, Bits=22025.05 | 19.97 | 28.07 | 27.05 | 53.75 | 50.04 | 35.78 | |
| MagRModel=LLaMA 3, Size=8B, Bits=2, extra coordinate descent iterations=true2025.05 | 18.43 | 30.09 | 27.6 | 55.98 | 50.12 | 36.45 |