Large Language Model Reasoning and Understanding on Llama-3.2-3B-Instruct (test val)
37.3MMLU ProBF16
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BF16Setting=Full Precision, Quantization Method=None2026.05 | 37.3 | 78.54 | 70.24 | 71.75 | 68.51 | 65.27 | 100 | |
| ScaleSweepMSESetting=WA, Quantization Method=GPTQ2026.05 | 35.09 | 74.68 | 65.62 | 70.26 | 68.75 | 62.88 | 96.34 | |
| ScaleSweepSetting=WA, Quantization Method=GPTQ2026.05 | 34.97 | 73.69 | 70.61 | 70.37 | 66.46 | 63.22 | 96.86 | |
| 4/6Setting=WA, Quantization Method=GPTQ2026.05 | 34.23 | 72.78 | 69.69 | 70.53 | 66.06 | 62.66 | 96 | |
| ScaleSweepSetting=WA, Quantization Method=RTN2026.05 | 34.12 | 73.16 | 68.76 | 70.18 | 68.98 | 63.04 | 96.58 | |
| 4/6Setting=WA, Quantization Method=RTN2026.05 | 34.1 | 71.95 | 67.28 | 70.11 | 65.51 | 61.79 | 94.67 | |
| ScaleSweepMSESetting=WA, Quantization Method=RTN2026.05 | 33.96 | 73.46 | 68.02 | 70.39 | 67.09 | 62.59 | 95.89 | |
| ScaleSweepSetting=WAKV, Quantization Method=GPTQ2026.05 | 33.75 | — | 67.65 | 70.07 | 67.25 | 62.1 | 95.15 | |
| ScaleSweepMSESetting=WAKV, Quantization Method=GPTQ2026.05 | 33.44 | 72.33 | 65.62 | 70.22 | 66.93 | 61.71 | 94.54 | |
| AbsMaxSetting=WA, Quantization Method=RTN2026.05 | 33.43 | 71.49 | 63.96 | 70.3 | 68.27 | 61.49 | 94.21 | |
| AbsMaxSetting=WA, Quantization Method=GPTQ2026.05 | 33.29 | 71.57 | 65.43 | 69.85 | 68.75 | 61.78 | 94.65 | |
| ScaleSweepSetting=WAKV, Quantization Method=RTN2026.05 | 33.1 | 72.33 | 67.84 | 70.08 | 67.8 | 62.23 | 95.34 | |
| ScaleSweepMSESetting=WAKV, Quantization Method=RTN2026.05 | 33.08 | 72.33 | 67.47 | 69.79 | 66.3 | 61.79 | 94.67 | |
| ScaleSweepSetting=WAKVQ, Quantization Method=GPTQ2026.05 | 33.02 | 71.11 | 65.99 | 69.43 | 66.3 | 61.17 | 93.72 | |
| 4/6Setting=WAKV, Quantization Method=GPTQ2026.05 | 32.41 | 71.11 | 66.17 | 69.5 | 67.4 | 61.32 | 93.95 | |
| ScaleSweepMSESetting=WAKVQ, Quantization Method=GPTQ2026.05 | 32.21 | 68.76 | 62.66 | 69.25 | 65.82 | 59.74 | 91.53 | |
| 4/6Setting=WAKV, Quantization Method=RTN2026.05 | 31.99 | 69.83 | 67.84 | 69.63 | 66.69 | 61.19 | 93.76 | |
| ScaleSweepSetting=WAKVQ, Quantization Method=RTN2026.05 | 31.93 | 69.9 | 66.91 | 69.73 | 66.38 | 60.97 | 93.41 | |
| ScaleSweepMSESetting=WAKVQ, Quantization Method=RTN2026.05 | 31.33 | 68.31 | 66.17 | 69.13 | 66.69 | 60.33 | 92.43 | |
| AbsMaxSetting=WAKV, Quantization Method=GPTQ2026.05 | 31.26 | 67.63 | 66.73 | 69.81 | 66.85 | 60.45 | 92.62 | |
| AbsMaxSetting=WAKV, Quantization Method=RTN2026.05 | 30.93 | 67.17 | 66.36 | 69.74 | 68.03 | 60.45 | 92.61 | |
| 4/6Setting=WAKVQ, Quantization Method=GPTQ2026.05 | 30.73 | 66.11 | 65.8 | 69.01 | 64.8 | 59.29 | 90.84 | |
| 4/6Setting=WAKVQ, Quantization Method=RTN2026.05 | 30.44 | 67.32 | 60.44 | 69.46 | 67.32 | 59 | 90.39 | |
| AbsMaxSetting=WAKVQ, Quantization Method=GPTQ2026.05 | 29.81 | 62.62 | 61.74 | 69.05 | 65.35 | 57.71 | 88.43 | |
| AbsMaxSetting=WAKVQ, Quantization Method=RTN2026.05 | 28 | 61.87 | 63.96 | 68.71 | 65.43 | 57.59 | 88.24 |