Zero-shot Evaluation Suite (ARC, Hellaswag, LAMBADA, PIQA, Winogrande)
73.39ARC-CFP16
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FP16Bits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 73.39 | 87.42 | 87.89 | 84.62 | 87.32 | 84.76 | 84.23 | |
| FP16Bits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 73.39 | 87.42 | 87.89 | 84.62 | 87.32 | 84.76 | 84.23 | |
| OursBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 67.41 | 85.97 | 85.21 | 80.17 | 85.62 | 81.49 | 80.97 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 65.98 | 84.87 | 84.08 | 78.32 | 84.87 | 80.39 | 79.75 | |
| FP16Bits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 64.51 | 82.63 | 83.42 | 79.44 | 83.76 | 75.63 | 78.23 | |
| FP16Bits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 64.51 | 82.63 | 83.42 | 79.44 | 83.76 | 75.63 | 78.23 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 64.25 | 85.94 | 84.93 | 79.37 | 84.44 | 80.74 | 79.95 | |
| FP16#Bits W-A-KV=3-3-3, Model=LLaMA3-70B2026.05 | 64.25 | 85.94 | 84.93 | 79.37 | 84.44 | 80.74 | 79.95 | |
| FP16#Bits W-A-KV=2-4-16, Model=LLaMA3-70B2026.05 | 64.25 | 85.94 | 84.93 | 79.37 | 84.44 | 80.74 | 79.95 | |
| FP16Bits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 64.25 | 85.94 | 84.93 | 79.37 | 84.44 | 80.74 | 79.95 | |
| FP16Bits (W-A-KV)=2-4-16, Model=LLaMA3-70B, Zero-shot=true2026.05 | 64.25 | 85.94 | 84.93 | 79.37 | 84.44 | 80.74 | 79.95 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 63.76 | 81.03 | 81.27 | 75.86 | 81.36 | 77.2 | 76.74 | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 62.83 | 84.88 | 84.07 | 79.42 | 84.01 | 79.56 | 79.13 | |
| OursBits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 62.83 | 84.88 | 84.07 | 79.42 | 84.01 | 79.56 | 79.19 | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 61.95 | 84.47 | 83.87 | 77.99 | 83.95 | 79.24 | 78.58 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 61.95 | 84.47 | 83.87 | 77.99 | 83.95 | 79.24 | 78.58 | |
| QuaRotBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 61.08 | 78.64 | 78.32 | 73.62 | 79.62 | 74.42 | 74.28 | |
| OursBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 59.76 | 78.81 | 77.89 | 76.63 | 79.64 | 73.98 | 74.45 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 58.64 | 76.88 | 76.49 | 75.31 | 79.38 | 73.42 | 73.35 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 57.71 | 81.02 | 83.81 | 79.6 | 82.7 | 77.98 | 77.05 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 57.38 | 74.2 | 75.55 | 74.36 | 78.83 | 70.76 | 71.84 | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 56.6 | 80.32 | 82.97 | 79.84 | 82.9 | 77.03 | 76.61 | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 56.4 | 80.09 | 82.91 | 80.01 | 82.92 | 76.87 | 76.53 | |
| BDQ#Bits W-A-KV=3-3-3, Model=LLaMA3-70B2026.05 | 56.12 | 81.22 | 79.63 | 74.79 | 80.14 | 75.67 | 74.59 | |
| QuaRotBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 55.67 | 73.64 | 72.97 | 71.93 | 76.37 | 68.18 | 69.79 | |
| QuaRot#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 55.46 | 79.76 | 81.58 | 79.35 | 81.83 | 76.09 | 75.68 | |
| SpinQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 55.38 | 79.04 | 82.57 | 78.75 | 82.37 | 78.22 | 76.06 | |
| OursBits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 54.76 | 82.07 | 76.64 | 73.52 | 68.93 | 68.92 | 70.81 | |
| FlatQuant#Bits W-A-KV=3-3-3, Model=LLaMA3-70B2026.05 | 54.37 | 80.31 | 78.67 | 73.57 | 79.03 | 73.37 | 73.22 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA3-8B2026.05 | 53.5 | 77.57 | 79.12 | 75.51 | 80.74 | 72.93 | 73.23 | |
| FP16#Bits W-A-KV=3-3-3, Model=LLaMA3-8B2026.05 | 53.5 | 77.57 | 79.12 | 75.51 | 80.74 | 72.93 | 73.23 | |
| FP16#Bits W-A-KV=2-4-16, Model=LLaMA3-8B2026.05 | 53.5 | 77.57 | 79.12 | 75.51 | 80.74 | 72.93 | 73.23 | |
| FP16Bits (W-A-KV)=4-4-4, Model=LLaMA3-8B, Zero-shot=true2026.05 | 53.5 | 77.57 | 79.12 | 75.51 | 80.74 | 72.93 | 73.23 | |
| FP16Bits (W-A-KV)=2-4-16, Model=LLaMA3-8B, Zero-shot=true2026.05 | 53.5 | 77.57 | 79.12 | 75.51 | 80.74 | 72.93 | 73.23 | |
| BDQ#Bits W-A-KV=2-4-16, Model=LLaMA3-70B2026.05 | 53.26 | 80.06 | 77.49 | 72.57 | 78.39 | 71.53 | 72.22 | |
| OursBits (W-A-KV)=2-4-16, Model=LLaMA3-70B, Zero-shot=true2026.05 | 53.26 | 80.06 | 77.49 | 72.57 | 78.39 | 71.53 | 72.22 | |
| FlatQuantBits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 52.78 | 80.12 | 74.03 | 71.77 | 66.73 | 66.74 | 68.69 | |
| SpinQuant#Bits W-A-KV=3-3-3, Model=LLaMA3-70B2026.05 | 52.28 | 78.24 | 76.61 | 72.18 | 77.37 | 70.78 | 71.24 | |
| SpinQuant#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 51.96 | 77.4 | 77.29 | 71.9 | 79.33 | 72.06 | 71.66 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 51.96 | 77.4 | 77.29 | 71.9 | 79.33 | 72.06 | 71.66 | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA3-8B2026.05 | 51.03 | 76.1 | 76.77 | 73.42 | 78.57 | 72.88 | 71.46 | |
| OursBits (W-A-KV)=4-4-4, Model=LLaMA3-8B, Zero-shot=true2026.05 | 51.03 | 76.1 | 76.77 | 73.42 | 78.57 | 72.88 | 71.46 | |
| FlatQuant#Bits W-A-KV=2-4-16, Model=LLaMA3-70B2026.05 | 50.71 | 78.61 | 75.82 | 70.93 | 75.42 | 68.68 | 70.02 | |
| FlatQuantBits (W-A-KV)=2-4-16, Model=LLaMA3-70B, Zero-shot=true2026.05 | 50.71 | 78.61 | 75.82 | 70.93 | 75.42 | 68.68 | 70.02 | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA3-8B2026.05 | 50.51 | 75.88 | 76.49 | 73.2 | 79 | 72.93 | 71.33 | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=LLaMA3-8B, Zero-shot=true2026.05 | 50.51 | 75.88 | 76.49 | 73.2 | 79 | 72.93 | 71.33 | |
| SpinQuantBits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 50.37 | 78.09 | 72.46 | 69.1 | 64.52 | 64.57 | 66.51 | |
| QuaRot#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 49.49 | 74.37 | 77.22 | 71.69 | 78.89 | 71.03 | 70.45 | |
| QuaRotBits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 49.49 | 74.37 | 77.22 | 71.69 | 78.89 | 71.03 | 70.45 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 49.15 | 77.44 | 79.39 | 76.73 | 80.47 | 72.14 | 72.55 | |
| SpinQuant#Bits W-A-KV=2-4-16, Model=LLaMA3-70B2026.05 | 48.74 | 76.74 | 63.01 | 67.79 | 73.82 | 65.93 | 66.01 | |
| SpinQuantBits (W-A-KV)=2-4-16, Model=LLaMA3-70B, Zero-shot=true2026.05 | 48.74 | 76.74 | 63.01 | 67.79 | 73.82 | 65.93 | 66.01 | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 48.52 | 76.87 | 77.9 | 76.47 | 79.83 | 70.77 | 71.73 | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 48.38 | 76.94 | 77.88 | 76.4 | 79.65 | 70.56 | 71.64 | |
| QuaRot#Bits W-A-KV=3-3-3, Model=LLaMA3-70B2026.05 | 47.86 | 74.31 | 70.53 | 67.57 | 72.09 | 67.53 | 66.64 | |
| SpinQuant#Bits W-A-KV=4-4-4, Model=LLaMA3-8B2026.05 | 47.27 | 74.2 | 74.55 | 70.29 | 77.37 | 68.51 | 68.7 | |
| SpinQuantBits (W-A-KV)=4-4-4, Model=LLaMA3-8B, Zero-shot=true2026.05 | 47.27 | 74.2 | 74.55 | 70.29 | 77.37 | 68.51 | 68.7 | |
| QuaRotBits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 46.37 | 74.3 | 70.05 | 64.07 | 62.07 | 61.87 | 63.12 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA2-7B2026.05 | 46.16 | 74.54 | 75.98 | 73.92 | 79.05 | 69.06 | 69.79 | |
| QuaRot#Bits W-A-KV=4-4-4, Model=LLaMA3-8B2026.05 | 45.73 | 70.83 | 72.97 | 62.7 | 75.35 | 67.17 | 65.79 | |
| QuaRotBits (W-A-KV)=4-4-4, Model=LLaMA3-8B, Zero-shot=true2026.05 | 45.73 | 70.83 | 72.97 | 62.7 | 75.35 | 67.17 | 65.79 | |
| SpinQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 43.69 | 72.43 | 75.52 | 72.42 | 78.4 | 68.9 | 68.56 | |
| OursBits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 43.43 | 66.83 | 65.61 | 57.98 | 72.62 | 62.38 | 61.47 | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA2-7B2026.05 | 43.07 | 73.09 | 73.36 | 72.06 | 77.57 | 67.9 | 67.84 | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-7B2026.05 | 43 | 71.21 | 73.31 | 72.06 | 77.53 | 67.72 | 67.47 | |
| QuaRot#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 42.83 | 69.95 | 73.54 | 65.62 | 77.69 | 67.88 | 66.25 | |
| QuaRot#Bits W-A-KV=2-4-16, Model=LLaMA3-70B2026.05 | 42.76 | 72.07 | 68.62 | 62.57 | 68.94 | 59.76 | 62.45 | |
| QuaRotBits (W-A-KV)=2-4-16, Model=LLaMA3-70B, Zero-shot=true2026.05 | 42.76 | 72.07 | 68.62 | 62.57 | 68.94 | 59.76 | 62.45 | |
| QuaRot#Bits W-A-KV=4-4-4, Model=LLaMA2-7B2026.05 | 42.32 | 68.35 | 72.53 | 65.4 | 76.33 | 65.11 | 65.01 | |
| SpinQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-7B2026.05 | 41.72 | 69.28 | 72.9 | 71.28 | 76.17 | 66.06 | 66.23 | |
| FlatQuantBits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 40.76 | 64.64 | 62.34 | 53.16 | 69.35 | 59.15 | 58.23 | |
| SpinQuantBits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 38.77 | 62.72 | 59.87 | 51.33 | 67.73 | 57.42 | 56.31 | |
| BDQ#Bits W-A-KV=3-3-3, Model=LLaMA3-8B2026.05 | 37.4 | 65.3 | 65.3 | 53.6 | 73.89 | 61.42 | 59.48 | |
| BDQ#Bits W-A-KV=2-4-16, Model=LLaMA3-8B2026.05 | 36.69 | 64.89 | 64.39 | 52.88 | 72.71 | 60.33 | 58.65 | |
| OursBits (W-A-KV)=2-4-16, Model=LLaMA3-8B, Zero-shot=true2026.05 | 36.69 | 64.89 | 64.39 | 52.88 | 72.71 | 60.33 | 58.65 | |
| FlatQuant#Bits W-A-KV=3-3-3, Model=LLaMA3-8B2026.05 | 35.41 | 63.26 | 65.3 | 52.49 | 73.56 | 60.69 | 58.45 | |
| QuaRotBits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-8B, Zero-shot=true2026.05 | 34.78 | 58.75 | 57.46 | 46.87 | 64.31 | 54.38 | 52.75 | |
| FlatQuant#Bits W-A-KV=2-4-16, Model=LLaMA3-8B2026.05 | 33.37 | 61.26 | 62.55 | 49.07 | 68.59 | 56.89 | 55.28 | |
| FlatQuantBits (W-A-KV)=2-4-16, Model=LLaMA3-8B, Zero-shot=true2026.05 | 33.37 | 61.26 | 62.55 | 49.07 | 68.59 | 56.89 | 55.28 | |
| SpinQuant#Bits W-A-KV=3-3-3, Model=LLaMA3-8B2026.05 | 32.55 | 61.03 | 63.59 | 49.8 | 71.29 | 57.93 | 56.03 | |
| SpinQuant#Bits W-A-KV=2-4-16, Model=LLaMA3-8B2026.05 | 31.77 | 58.93 | 61.29 | 47.36 | 66.25 | 55.42 | 53.5 | |
| SpinQuantBits (W-A-KV)=2-4-16, Model=LLaMA3-8B, Zero-shot=true2026.05 | 31.77 | 58.93 | 61.29 | 47.36 | 66.25 | 55.42 | 53.5 | |
| QuaRot#Bits W-A-KV=3-3-3, Model=LLaMA3-8B2026.05 | 28.93 | 57.42 | 60.33 | 45.81 | 66.34 | 54.25 | 52.18 | |
| QuaRot#Bits W-A-KV=2-4-16, Model=LLaMA3-8B2026.05 | 28.59 | 54.76 | 54.62 | 41.9 | 60.03 | 51.33 | 48.53 | |
| QuaRotBits (W-A-KV)=2-4-16, Model=LLaMA3-8B, Zero-shot=true2026.05 | 28.59 | 54.76 | 54.62 | 41.9 | 60.03 | 51.33 | 48.53 |