Commonsense Reasoning on Six common sense reasoning tasks (Zero-shot)
77.51Average Accuracy (Zero-shot)FPTQuant
Evaluation Results
| Method | Links | |
|---|---|---|
| FPTQuantBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.06 | 77.51 | |
| QuaRot-optBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.06 | 76.24 | |
| FlatQuantBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.06 | 76.22 | |
| OSTQuantBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.06 | 75.96 | |
| SpinQuantBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.06 | 75.51 | |
| FP16Bits (W-A-KV)=16-16-16, Model=Q2.5-32B, Zero-shot=true2025.06 | 75.29 | |
| Full Precision BaselinePrecision=W16A16, Model=Qwen3 32B, Zero-shot=true2026.06 | 75.2 | |
| RTN-optBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.06 | 75.15 | |
| FP16Bits (W-A-KV)=16-16-16, Model=M-8B it, Zero-shot=true2025.06 | 74.37 | |
| Full Precision BaselinePrecision=W16A16, Model=Qwen3 14B, Zero-shot=true2026.06 | 74.2 | |
| MosaicQuantPrecision=W4A8, Model=Qwen3 32B, Zero-shot=true2026.06 | 74.2 | |
| FPTQuantBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.06 | 73.59 | |
| Full Precision BaselinePrecision=W16A16, Model=LLaMA3 8B, Zero-shot=true2026.06 | 73.5 | |
| QuaRot-optBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.06 | 73.41 | |
| FlatQuantBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.06 | 73.41 | |
| MosaicQuantPrecision=W4A8, Model=Qwen3 14B, Zero-shot=true2026.06 | 73.4 | |
| FP16Bits (W-A-KV)=16-16-16, Model=L3-8B, Zero-shot=true2025.06 | 73.33 | |
| MosaicQuantPrecision=W4A4, Model=Qwen3 32B, Zero-shot=true2026.06 | 73.3 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.06 | 73.05 | |
| FPTQuantBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.06 | 72.92 | |
| MosaicQuantPrecision=W4A4, Model=Qwen3 14B, Zero-shot=true2026.06 | 72.9 | |
| OSTQuantBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.06 | 72.87 | |
| FPTQuantBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.06 | 72.72 | |
| MosaicQuantPrecision=W4A8, Model=LLaMA3 8B, Zero-shot=true2026.06 | 72.7 | |
| SpinQuantPrecision=W4A4, Model=Qwen3 32B, Zero-shot=true2026.06 | 72.7 | |
| SpinQuantBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.06 | 72.48 | |
| AtomPrecision=W4A4, Model=Qwen3 32B, Zero-shot=true2026.06 | 72.2 | |
| FlatQuantBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.06 | 72.11 | |
| ResQPrecision=W4A4, Model=Qwen3 32B, Zero-shot=true2026.06 | 72 | |
| Full Precision BaselinePrecision=W16A16, Model=Qwen3 8B, Zero-shot=true2026.06 | 71.6 | |
| SpinQuantBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.06 | 71.6 | |
| OSTQuantBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.06 | 71.46 | |
| MosaicQuantPrecision=W4A4, Model=LLaMA3 8B, Zero-shot=true2026.06 | 71.2 | |
| MosaicQuantPrecision=W4A8, Model=Qwen3 8B, Zero-shot=true2026.06 | 70.6 | |
| RTNBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.06 | 70.28 | |
| FP16Bits (W-A-KV)=16-16-16, Model=L2-7B, Zero-shot=true2025.06 | 69.79 | |
| Full Precision BaselinePrecision=W16A16, Model=Qwen3 4B, Zero-shot=true2026.06 | 69.4 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.06 | 69.12 | |
| MosaicQuantPrecision=W4A8, Model=Qwen3 4B, Zero-shot=true2026.06 | 68.7 | |
| MosaicQuantPrecision=W4A4, Model=Qwen3 8B, Zero-shot=true2026.06 | 68.6 | |
| QuaRot-optBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.06 | 68.01 | |
| QuaRot-optBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.06 | 67.6 | |
| RTN-optBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.06 | 67.35 | |
| OSTQuantBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.06 | 66.84 | |
| MosaicQuantPrecision=W4A4, Model=Qwen3 4B, Zero-shot=true2026.06 | 66.3 | |
| Full Precision BaselinePrecision=W16A16, Model=LLaMA3.2 3B, Zero-shot=true2026.06 | 66.2 | |
| SpinQuantBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.06 | 66.01 | |
| FPTQuantBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.06 | 65.96 | |
| FP16Bits (W-A-KV)=16-16-16, Model=L3.2-3B it, Zero-shot=true2025.06 | 65.63 | |
| MosaicQuantPrecision=W4A8, Model=LLaMA3.2 3B, Zero-shot=true2026.06 | 65.3 | |
| FlatQuantBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.06 | 65.04 | |
| MosaicQuantPrecision=W4A4, Model=LLaMA3.2 3B, Zero-shot=true2026.06 | 64.9 | |
| QuaRot-optBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.06 | 64.68 | |
| FPTQuantBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.06 | 64 | |
| FPTQuantBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.06 | 63.69 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.06 | 63.51 | |
| QuaRot-optBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.06 | 63.43 | |
| SpinQuantBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.06 | 63.28 | |
| QuaRot-optBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.06 | 63.12 | |
| AWQPrecision=W4A16, Model=LLaMA3.2 3B, Zero-shot=true2026.06 | 63.1 | |
| OSTQuantBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.06 | 62.48 | |
| FlatQuantBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.06 | 62.07 | |
| OSTQuantBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.06 | 61.85 | |
| GPTQPrecision=W4A16, Model=LLaMA3.2 3B, Zero-shot=true2026.06 | 61.7 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.06 | 61.43 | |
| RTN-optBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.06 | 61.09 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.06 | 61 | |
| RTNPrecision=W4A16, Model=LLaMA3.2 3B, Zero-shot=true2026.06 | 60.9 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.06 | 60.6 | |
| FPTQuantBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.06 | 59.46 | |
| RTN-optBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.06 | 56.93 | |
| OSTQuantBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.06 | 56.69 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.06 | 54.88 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.06 | 54.58 | |
| QuaRot-optBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.06 | 54.38 | |
| FPTQuantBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.06 | 52.96 | |
| OSTQuantBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.06 | 52.43 | |
| RTN-optBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.06 | 51.97 | |
| OSTQuantBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.06 | 50.88 | |
| RTNBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.06 | 47.75 | |
| RTNBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.06 | 47.27 | |
| RTNBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.06 | 45 | |
| QuaRot-optBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.06 | 42.76 | |
| RTNBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.06 | 39.13 | |
| RTNBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.06 | 37.67 | |
| RTN-optBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.06 | 36.92 | |
| RTN-optBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.06 | 31.16 | |
| RTNBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.06 | 31.02 | |
| OSTQuantBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.06 | 30.75 | |
| QuaRot-optBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.06 | 30.21 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.06 | 30.17 | |
| RTN-optBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.06 | 30.04 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.06 | 29.9 | |
| RTNBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.06 | 29.83 | |
| FPTQuantBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.06 | 29.76 | |
| RTN-optBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.06 | 29.63 | |
| RTN-optBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.06 | 29.54 | |
| RTNBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.06 | 29.33 | |
| RTNBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.06 | 29.17 |