Question Answering on PIQA, ARC-e, ARC-c, BoolQ, HellaSwag, WinoGrande (Accuracy & Efficiency)
84.39PIQA AccuracyGAMMA
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GAMMAModel=Qwen3-32B, Evaluation Protocol=Zero-shot, Bit-width=3.0bit2026.05 | 84.39 | 93.83 | 89.83 | 87.49 | 86.38 | 73.95 | 85.97 | — | — | — | |
| GAMMAModel=Qwen3-32B, Evaluation Protocol=Zero-shot, Bit-width=2.5bit2026.05 | 83.57 | 91.53 | 85.08 | 88.47 | 85.82 | 73.32 | 84.63 | — | — | — | |
| QTIPModel=Qwen3-32B, Evaluation Protocol=Zero-shot, Bit-width=3bit2026.05 | 83.35 | 93.12 | 86.1 | 87.55 | 86.57 | 74.11 | 85.13 | — | — | — | |
| GAMMAModel=Qwen3-32B, Evaluation Protocol=Zero-shot, Bit-width=3.5bit2026.05 | 83.3 | 93.47 | 89.83 | 87.55 | 87.69 | 76.09 | 86.32 | — | — | — | |
| 16bitModel=Qwen3-32B, Evaluation Protocol=Zero-shot, Precision=16bit2026.05 | 82.75 | 90.3 | 85.76 | 87.86 | 86.39 | 75.93 | 84.83 | — | — | — | |
| FP16Backbone=QWEN-3-32B, Type=N/A2026.06 | 82.1 | 83.16 | 60.92 | 86.36 | 82.58 | 72.92 | 78.01 | — | — | 100 | |
| LLaMA3-8BSpike=✗, Zero-shot=true2026.04 | 80.85 | 77.78 | 53.41 | 81.28 | 79.16 | 72.84 | 74.22 | 7.97 | 36.67 | — | |
| FP16Backbone=QWEN-3-14B, Type=N/A2026.06 | 79.87 | 82.83 | 60.15 | 89.3 | 78.84 | 72.85 | 77.31 | — | — | 100 | |
| QUIP#Backbone=QWEN-3-32B, Type=VECTOR, Bits=2-bit2026.06 | 79.7 | 80 | 58.1 | 87.89 | 78.46 | 73.63 | 76.3 | — | — | 98 | |
| UniSVQBackbone=QWEN-3-32B, Type=UNISVQ, Bits=2-bit2026.06 | 78.83 | 80.81 | 58.44 | 87.89 | 77.07 | 73.88 | 76.15 | — | — | 98 | |
| AQLMBackbone=QWEN-3-32B, Type=VECTOR, Bits=2-bit2026.06 | 77.8 | 80.85 | 58.27 | 86.94 | 76.21 | 72.13 | 75.37 | — | — | 97 | |
| FP16Backbone=QWEN-3-8B, Type=N/A2026.06 | 77.48 | 80.89 | 56.4 | 86.64 | 74.96 | 68.35 | 74.12 | — | — | 100 | |
| QUIP#Backbone=QWEN-3-14B, Type=VECTOR, Bits=2-bit2026.06 | 76.99 | 76.89 | 53.67 | 87.95 | 71.91 | 71.67 | 73.18 | — | — | 95 | |
| OSTQUANTBackbone=QWEN-3-32B, Type=SCALAR, Bits=2-bit2026.06 | 76.5 | 67.55 | 46.84 | 80.8 | 68.37 | 69.69 | 68.29 | — | — | 88 | |
| UniSVQBackbone=QWEN-3-14B, Type=UNISVQ, Bits=2-bit2026.06 | 76.5 | 78.49 | 51.79 | 86.82 | 69.96 | 71.27 | 72.47 | — | — | 94 | |
| AQLMBackbone=QWEN-3-14B, Type=VECTOR, Bits=2-bit2026.06 | 76.17 | 75.29 | 50.6 | 87.34 | 69.17 | 70.8 | 71.56 | — | — | 93 | |
| SDLLMSpike=✓, Bit=4-1.58, T x D=1 × 8, R=0.210, Zero-shot=true2026.04 | 75.9 | 67.05 | 44.37 | 72.45 | 73.26 | 67.01 | 66.67 | 0.84 | 0.75 | — | |
| FP16Backbone=QWEN-3-4B, Type=N/A2026.06 | 75.84 | 81.23 | 58.36 | 84.68 | 69.06 | 68.11 | 72.88 | — | — | 100 | |
| DuQuantSpike=✗, Bit=4-4, Zero-shot=true2026.04 | 75.68 | 68.48 | 41.81 | 71.99 | 73.07 | 66.22 | 66.21 | 1 | 4.58 | — | |
| QUIP#Backbone=QWEN-3-8B, Type=VECTOR, Bits=2-bit2026.06 | 74.32 | 68.43 | 46.5 | 83.12 | 66.62 | 66.3 | 67.55 | — | — | 91 | |
| UniSVQBackbone=QWEN-3-8B, Type=UNISVQ, Bits=2-bit2026.06 | 74.16 | 72.35 | 45.82 | 85.07 | 63.18 | 67.09 | 67.95 | — | — | 92 | |
| OSTQUANTBackbone=QWEN-3-14B, Type=SCALAR, Bits=2-bit2026.06 | 73.72 | 69.61 | 43.26 | 84.74 | 59.17 | 66.93 | 66.24 | — | — | 86 | |
| AQLMBackbone=QWEN-3-8B, Type=VECTOR, Bits=2-bit2026.06 | 73.07 | 72.31 | 45.22 | 73.73 | 60.99 | 64.33 | 64.94 | — | — | 88 | |
| QUIP#Backbone=QWEN-3-4B, Type=VECTOR, Bits=2-bit2026.06 | 73.01 | 70.41 | 45.82 | 79.72 | 58.55 | 62.75 | 65.04 | — | — | 89 | |
| SPINQUANTBackbone=QWEN-3-32B, Type=SCALAR, Bits=2-bit2026.06 | 72.91 | 63.89 | 43.34 | 84.4 | 67.4 | 67.96 | 66.65 | — | — | 85 | |
| UniSVQBackbone=QWEN-3-4B, Type=UNISVQ, Bits=2-bit2026.06 | 70.89 | 65.82 | 43.34 | 82.26 | 55.27 | 62.35 | 63.32 | — | — | 87 | |
| SPINQUANTBackbone=QWEN-3-14B, Type=SCALAR, Bits=2-bit2026.06 | 70.4 | 63.97 | 38.99 | 79.42 | 56.28 | 65.51 | 62.43 | — | — | 81 | |
| QUIPBackbone=QWEN-3-32B, Type=SCALAR, Bits=2-bit2026.06 | 69.2 | 44.57 | 34.89 | 53.14 | 62.24 | 53.35 | 52.9 | — | — | 68 | |
| AQLMBackbone=QWEN-3-4B, Type=VECTOR, Bits=2-bit2026.06 | 68.61 | 66.88 | 40.96 | 77.22 | 53.02 | 60.85 | 61.26 | — | — | 84 | |
| OSTQUANTBackbone=QWEN-3-8B, Type=SCALAR, Bits=2-bit2026.06 | 68.5 | 60.02 | 34.64 | 73.21 | 50.29 | 58.17 | 57.47 | — | — | 78 | |
| QUIPBackbone=QWEN-3-14B, Type=SCALAR, Bits=2-bit2026.06 | 66.87 | 46.42 | 29.86 | 61.22 | 57.47 | 57.14 | 53.16 | — | — | 69 | |
| SPINQUANTBackbone=QWEN-3-8B, Type=SCALAR, Bits=2-bit2026.06 | 63.17 | 48.4 | 30.46 | 67.06 | 47.94 | 58.64 | 52.61 | — | — | 71 | |
| AtomSpike=✗, Bit=4-4, Zero-shot=true2026.04 | 62.95 | 49.45 | 30.12 | 60.31 | 53.75 | 56.04 | 52.1 | 1 | 4.58 | — | |
| QUIPBackbone=QWEN-3-8B, Type=SCALAR, Bits=2-bit2026.06 | 59.9 | 31.69 | 24.91 | 54.89 | 41.41 | 50.12 | 43.82 | — | — | 59 | |
| SPINQUANTBackbone=QWEN-3-4B, Type=SCALAR, Bits=2-bit2026.06 | 59.19 | 41.05 | 28.92 | 64.34 | 40.74 | 51.78 | 47.67 | — | — | 65 | |
| QUIPBackbone=QWEN-3-4B, Type=SCALAR, Bits=2-bit2026.06 | 57.67 | 35.82 | 23.98 | 48.29 | 37.38 | 49.41 | 42.09 | — | — | 58 | |
| OSTQUANTBackbone=QWEN-3-4B, Type=SCALAR, Bits=2-bit2026.06 | 57.62 | 32.78 | 29.35 | 62.17 | 34.66 | 53.43 | 45 | — | — | 62 | |
| SmoothQuantSpike=✗, Bit=4-4, Zero-shot=true2026.04 | 54.57 | 31.9 | 24.23 | 52.72 | 31.26 | 51.14 | 40.97 | 1 | 4.58 | — | |
| GPTQBackbone=QWEN-3-8B, Type=SCALAR, Bits=2-bit2026.06 | 52.5 | 25.67 | 26.79 | 42.87 | 25.84 | 50.04 | 37.29 | — | — | 50 | |
| GPTQBackbone=QWEN-3-4B, Type=SCALAR, Bits=2-bit2026.06 | 51.52 | 26.05 | 26.62 | 43.3 | 26.12 | 48.3 | 36.99 | — | — | 51 | |
| AffineQuantSpike=✗, Bit=4-4, Zero-shot=true2026.04 | 50.71 | 25.93 | 26.02 | 40.55 | 26.07 | 48.46 | 36.29 | 1 | 4.58 | — | |
| GPTQBackbone=QWEN-3-32B, Type=SCALAR, Bits=2-bit2026.06 | 50.59 | 25.46 | 25.34 | 40.58 | 25.19 | 49.4 | 36.09 | — | — | 46 | |
| GPTQBackbone=QWEN-3-14B, Type=SCALAR, Bits=2-bit2026.06 | 50.27 | 25.76 | 25.94 | 42.02 | 24.76 | 47.2 | 37.29 | — | — | 48 | |
| OmniQuantSpike=✗, Bit=4-4, Zero-shot=true2026.04 | 50.22 | 26.94 | 24.57 | 37.98 | 26.55 | 50.2 | 36.08 | 1 | 4.58 | — |