Zero-shot Reasoning on Reasoning Suite (ARC-e, ARC-c, HellaSwag, PIQA, Winogrande)
6,540Average Reasoning ScoreRIA+SQ+VC+EBFT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RIA+SQ+VC+EBFTOutliers=16:256, Sparsity=8:16, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 6,540 | — | — | — | — | — | — | |
| RIA+SQOutliers=16:256, Sparsity=8:16, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 6,523 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=16:256, Sparsity=8:16, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 6,513 | — | — | — | — | — | — | |
| RIA+SQOutliers=16:256, Sparsity=8:16, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 6,503 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=16:256, Sparsity=2:4, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 6,401 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=16:256, Sparsity=2:4, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 6,334 | — | — | — | — | — | — | |
| RIA+SQOutliers=16:256, Sparsity=2:4, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 6,303 | — | — | — | — | — | — | |
| RIA+SQOutliers=16:256, Sparsity=2:4, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 6,296 | — | — | — | — | — | — | |
| FP16Model=3-70, type=-, bits=-2026.06 | 75.33 | 86.99 | 60.41 | 66.36 | 82.37 | 80.51 | — | |
| LiftUQModel=3-70, type=UQ, bits=3.022026.06 | 74.25 | 85.86 | 58.87 | 65.32 | 82.43 | 78.77 | — | |
| AWQ-g128Model=3-70, type=UQ, bits=3.132026.06 | 73.65 | 84.51 | 58.36 | 64.26 | 82.26 | 78.85 | — | |
| EPTQ-g128Model=3-70, type=UQ, bits=3.132026.06 | 72.42 | 83.12 | 55.12 | 65.53 | 80.52 | 77.82 | — | |
| FP16Model=2-70, type=-, bits=-2026.06 | 72.41 | 82.7 | 54.44 | 64.77 | 82.15 | 77.98 | — | |
| QuIP#Model=2-70, type=VQ, bits=3.002026.06 | 72.13 | 82.11 | 55.89 | 64.22 | 82.21 | 76.24 | — | |
| LiftUQModel=2-70, type=UQ, bits=3.022026.06 | 71.96 | 82.58 | 54.61 | 63.98 | 81.5 | 77.11 | — | |
| BF16Bits=BF162026.03 | 71.64 | 81.06 | 56.48 | 74.96 | 77.69 | 68.03 | 100 | |
| BATQuantBits=W4A8KV162026.03 | 71.14 | 79.92 | 56.14 | 73.1 | 77.97 | 68.59 | 99.34 | |
| GPTQBits=W4A8KV162026.03 | 70.81 | 80.6 | 55.89 | 73.16 | 77.31 | 67.09 | 98.82 | |
| RTNBits=W4A8KV162026.03 | 70.77 | 80.81 | 55.72 | 73.29 | 77.09 | 66.93 | 98.75 | |
| GPTQBits=W4A8KV82026.03 | 70.74 | 80.68 | 56.06 | 72.95 | 77.53 | 66.46 | 98.72 | |
| BATQuantBits=W4A8KV82026.03 | 70.57 | 79.8 | 55.63 | 73.15 | 77.09 | 67.17 | 98.5 | |
| RTNBits=W4A8KV82026.03 | 70.51 | 80.51 | 55.72 | 72.86 | 76.55 | 66.93 | 98.42 | |
| SmoothQuantBits=W4A8KV82026.03 | 70.28 | 79.21 | 55.03 | 72.76 | 76.99 | 67.4 | 98.08 | |
| FlatQuantBits=W4A8KV162026.03 | 70.21 | 79.63 | 55.72 | 72.66 | 76.82 | 66.22 | 98.01 | |
| QuaRotBits=W4A8KV82026.03 | 70.17 | 79.84 | 55.38 | 72.54 | 76.88 | 66.22 | 97.92 | |
| SmoothQuantBits=W4A8KV162026.03 | 70.12 | 79.04 | 55.8 | 72.38 | 76.55 | 66.85 | 97.93 | |
| BRQBits=W4A8KV162026.03 | 69.88 | 78.87 | 53.67 | 73.27 | 76.66 | 66.93 | 97.43 | |
| SpinQuantBits=W4A8KV162026.03 | 69.87 | 76.98 | 54.69 | 72.76 | 78.07 | 66.85 | 97.52 | |
| BRQBits=W4A8KV82026.03 | 69.77 | 78.11 | 52.99 | 73.09 | 76.88 | 67.8 | 97.26 | |
| QuaRotBits=W4A8KV162026.03 | 69.73 | 78.7 | 55.2 | 72.77 | 76.88 | 65.11 | 97.31 | |
| FlatQuantBits=W4A8KV82026.03 | 69.48 | 77.1 | 52.56 | 72.46 | 77.09 | 68.19 | 96.86 | |
| SpinQuantBits=W4A8KV82026.03 | 69.43 | 77.65 | 53.5 | 72.56 | 77.53 | 65.9 | 96.8 | |
| Baseline (Uncompressed)Model=LLaMA3-8B, Zero-shot=true2025.07 | 69.23 | — | — | — | — | — | — | |
| BATQuantBits=W4A4KV162026.03 | 68.7 | 77.53 | 53.33 | 71.12 | 75.3 | 66.22 | 95.84 | |
| BATQuantBits=W4A8KV42026.03 | 68.65 | 78.54 | 53.33 | 69.53 | 76.66 | 65.19 | 95.71 | |
| FP16Model=3-8, type=-, bits=-2026.06 | 68.58 | 80.09 | 50.43 | 60.17 | 79.6 | 72.61 | — | |
| Baseline (Uncompressed)Model=Mistral, Zero-shot=true2025.07 | 68.55 | — | — | — | — | — | — | |
| GPTQBits=W4A8KV42026.03 | 68.25 | 76.52 | 52.39 | 71.25 | 75.73 | 65.35 | 95.15 | |
| GPTQBits=W4A4KV162026.03 | 67.82 | 76.98 | 51.28 | 70.47 | 75.79 | 64.56 | 94.44 | |
| FP16Model=2-13, type=-, bits=-2026.06 | 67.81 | 79.42 | 48.29 | 60.07 | 79.05 | 72.22 | — | |
| RTNBits=W4A4KV162026.03 | 67.75 | 76.89 | 52.47 | 70.44 | 74.16 | 64.8 | 94.49 | |
| FlatQuantBits=W4A4KV162026.03 | 67.67 | 78.2 | 50.6 | 70.36 | 75.63 | 63.54 | 94.13 | |
| RTNBits=W4A8KV42026.03 | 67.62 | 76.89 | 51.96 | 70.54 | 75.08 | 63.61 | 94.22 | |
| FlatQuantBits=W4A8KV42026.03 | 67.46 | 77.9 | 52.13 | 69.6 | 75.14 | 62.51 | 93.97 | |
| LiftUQModel=3-8, type=UQ, bits=3.022026.06 | 67.3 | 78.83 | 46.59 | 58.42 | 78.73 | 73.95 | — | |
| QuaRotBits=W4A8KV42026.03 | 67.29 | 76.47 | 52.73 | 70.15 | 74.81 | 62.27 | 93.82 | |
| SmoothQuantBits=W4A4KV162026.03 | 66.99 | 75.72 | 50.09 | 70.15 | 74.37 | 64.64 | 93.29 | |
| SmoothQuantBits=W4A8KV42026.03 | 66.93 | 73.23 | 49.74 | 69.61 | 75.24 | 66.85 | 93.28 | |
| BRQBits=W4A8KV42026.03 | 66.82 | 75.97 | 50.68 | 70.38 | 74.65 | 62.43 | 93.04 | |
| LiftUQModel=2-13, type=UQ, bits=3.022026.06 | 66.67 | 77.99 | 46.25 | 59.16 | 78.84 | 71.11 | — | |
| SpinQuantBits=W4A8KV42026.03 | 66.49 | 74.07 | 49.32 | 69.82 | 75.95 | 63.3 | 92.53 | |
| VPTQModel=2-13, type=VQ, bits=3.032026.06 | 66.37 | 78.83 | 46.5 | 58.5 | 78.18 | 69.85 | — | |
| VPTQModel=3-8, type=VQ, bits=3.032026.06 | 66.32 | 78.45 | 44.8 | 57.85 | 78.78 | 71.74 | — | |
| QuIP#Model=2-13, type=VQ, bits=3.002026.06 | 66.26 | 77.9 | 44.62 | 58.26 | 78.07 | 72.45 | — | |
| BRQBits=W4A4KV162026.03 | 66.24 | 74.71 | 48.55 | 68.79 | 75.24 | 63.93 | 92.14 | |
| QuaRotBits=W4A4KV162026.03 | 65.86 | 74.28 | 50.43 | 67.55 | 73.67 | 63.38 | 91.81 | |
| RIA+SQ+EBFTModel=Mistral, Outliers=16:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 65.27 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTModel=LLaMA3-8B, Outliers=16:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 64.89 | — | — | — | — | — | — | |
| RIA+SQModel=Mistral, Outliers=16:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 64.87 | — | — | — | — | — | — | |
| FP16Model=2-7, type=-, bits=-2026.06 | 64.85 | 76.26 | 43.52 | 57.16 | 78.07 | 69.22 | — | |
| RIA+SQOutliers=8:256, Sparsity=8:16, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 64.52 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=4:256, Sparsity=8:16, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 64.52 | — | — | — | — | — | — | |
| RIA+SQ+EBFTModel=Mistral, Outliers=8:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 64.48 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=8:256, Sparsity=8:16, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 64.16 | — | — | — | — | — | — | |
| RIA+SQModel=Mistral, Outliers=8:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 64.09 | — | — | — | — | — | — | |
| RIA+SQ+EBFTModel=Mistral, Outliers=4:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 64.07 | — | — | — | — | — | — | |
| RIA+SQOutliers=8:256, Sparsity=8:16, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 64.03 | — | — | — | — | — | — | |
| RIA+SQ+VCModel=LLaMA3-8B, Outliers=16:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 63.98 | — | — | — | — | — | — | |
| RIA+SQOutliers=4:256, Sparsity=8:16, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 63.94 | — | — | — | — | — | — | |
| RIA+SQModel=Mistral, Outliers=4:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 63.81 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=8:256, Sparsity=8:16, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 63.76 | — | — | — | — | — | — | |
| RIA+SQModel=LLaMA3-8B, Outliers=16:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 63.75 | — | — | — | — | — | — | |
| LiftUQModel=2-7, type=UQ, bits=3.022026.06 | 63.71 | 75.07 | 41.02 | 56.57 | 77.89 | 67.97 | — | |
| RIA+SQOutliers=4:256, Sparsity=8:16, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 63.7 | — | — | — | — | — | — | |
| RIA+SQ+EBFTModel=Mistral, Outliers=-, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 63.63 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=4:256, Sparsity=8:16, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 63.63 | — | — | — | — | — | — | |
| SpinQuantBits=W4A4KV162026.03 | 63.53 | 68.18 | 45.65 | 67.41 | 74.21 | 62.19 | 88.36 | |
| QuIP#Model=2-7, type=VQ, bits=3.002026.06 | 63.52 | 74.62 | 41.89 | 55.85 | 77.04 | 68.19 | — | |
| RIA+SQ+VC+EBFTModel=LLaMA3-8B, Outliers=8:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 63.2 | — | — | — | — | — | — | |
| RIA+SQModel=Mistral, Outliers=-, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 63.04 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTModel=LLaMA3-8B, Outliers=4:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 62.71 | — | — | — | — | — | — | |
| RIA+SQ+VCModel=LLaMA3-8B, Outliers=8:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 62.29 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=8:256, Sparsity=2:4, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 62.18 | — | — | — | — | — | — | |
| RIA+SQ+EBFTModel=Mistral, Outliers=16:256, Sparsity=2:4, Calibration Set=WikiText2, Zero-shot=true2025.07 | 62.11 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTModel=LLaMA3-8B, Outliers=-, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 62.01 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=8:256, Sparsity=2:4, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 61.84 | — | — | — | — | — | — | |
| RIA+SQModel=LLaMA3-8B, Outliers=8:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 61.81 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTModel=LLaMA3-8B, Outliers=16:256, Sparsity=2:4, Calibration Set=WikiText2, Zero-shot=true2025.07 | 61.75 | — | — | — | — | — | — | |
| VPTQModel=2-7, type=VQ, bits=3.022026.06 | 61.7 | 69.1 | 39.3 | 54.9 | 77.3 | 68 | — | |
| RIA+SQ+VC+EBFTOutliers=4:256, Sparsity=2:4, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 61.5 | — | — | — | — | — | — | |
| RIA+SQModel=Mistral, Outliers=16:256, Sparsity=2:4, Calibration Set=WikiText2, Zero-shot=true2025.07 | 61.38 | — | — | — | — | — | — | |
| RIA+SQ+VC+EBFTOutliers=4:256, Sparsity=2:4, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 61.38 | — | — | — | — | — | — | |
| RIA+SQOutliers=8:256, Sparsity=2:4, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 61.19 | — | — | — | — | — | — | |
| RIA+SQ+VCModel=LLaMA3-8B, Outliers=4:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 60.98 | — | — | — | — | — | — | |
| RIA+SQOutliers=8:256, Sparsity=2:4, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 60.92 | — | — | — | — | — | — | |
| RIA+SQ+EBFTModel=Mistral, Outliers=8:256, Sparsity=2:4, Calibration Set=WikiText2, Zero-shot=true2025.07 | 60.68 | — | — | — | — | — | — | |
| RIA+SQModel=LLaMA3-8B, Outliers=4:256, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 60.6 | — | — | — | — | — | — | |
| RIA+SQOutliers=4:256, Sparsity=2:4, Calibration Set=WikiText2, Model=LLaMA2-13B2025.07 | 60.28 | — | — | — | — | — | — | |
| RIA+SQ+VCModel=LLaMA3-8B, Outliers=-, Sparsity=8:16, Calibration Set=WikiText2, Zero-shot=true2025.07 | 60.19 | — | — | — | — | — | — | |
| RIA+SQOutliers=4:256, Sparsity=2:4, Calibration Set=C4, Model=LLaMA2-13B2025.07 | 60.18 | — | — | — | — | — | — |