Large Language Model Evaluation on MMLU Pro, GSM8k, IFEval, HellaS, and WinoG
61.99MMLU ProBF16
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BF16Model=Qwen3-8B, Evaluation Mode=non-thinking2026.05 | 61.99 | 91.28 | 81.7 | 74.91 | 68.11 | 75.6 | 100 | |
| ScaleSweepMSEModel=Qwen3-8B, Setting=WA, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 59.67 | 89.23 | 82.62 | 73.66 | 67.64 | 74.57 | 98.63 | |
| ScaleSweepModel=Qwen3-8B, Setting=WA, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 59.57 | 90.37 | 83.18 | 73.68 | 69.3 | 75.22 | 99.5 | |
| 4/6Model=Qwen3-8B, Setting=WA, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 59.38 | 89.61 | 81.15 | 73.37 | 68.03 | 74.31 | 98.3 | |
| AbsMaxModel=Qwen3-8B, Setting=WA, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 58.78 | 89.31 | 81.33 | 73.2 | 68.19 | 74.16 | 98.1 | |
| AbsMaxModel=Qwen3-8B, Setting=WA, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 58.29 | 88.78 | 80.41 | 73.5 | 66.77 | 73.55 | 97.29 | |
| ScaleSweepModel=Qwen3-8B, Setting=WAKV, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 58.05 | 89.61 | 83.73 | 73.79 | 65.9 | 74.22 | 98.17 | |
| ScaleSweepModel=Qwen3-8B, Setting=WA, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 57.95 | 89.31 | 82.07 | 73.41 | 67.8 | 74.11 | 98.03 | |
| BF16Model=Qwen3-4B, Evaluation Mode=non-thinking2026.05 | 57.86 | 88.25 | 79.85 | 68.32 | 66.06 | 72.07 | 100 | |
| ScaleSweepMSEModel=Qwen3-8B, Setting=WAKV, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 57.5 | 89.16 | 81.7 | 73.5 | 65.98 | 73.57 | 97.31 | |
| ScaleSweepModel=Qwen3-8B, Setting=WAKVQ, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 57.34 | 88.86 | 80.96 | 72.66 | 65.98 | 73.16 | 96.77 | |
| ScaleSweepMSEModel=Qwen3-8B, Setting=WA, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 56.91 | 89.99 | 82.44 | 73.97 | 67.96 | 74.25 | 98.22 | |
| ScaleSweepMSEModel=Qwen3-8B, Setting=WAKVQ, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 56.81 | 88.7 | 81.7 | 72.84 | 66.69 | 73.35 | 97.02 | |
| AbsMaxModel=Qwen3-8B, Setting=WAKV, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 56.48 | 86.81 | 80.59 | 72.34 | 68.59 | 72.96 | 96.51 | |
| 4/6Model=Qwen3-8B, Setting=WAKV, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 56.09 | 89.46 | 80.59 | 73.54 | 66.14 | 73.17 | 96.78 | |
| ScaleSweepModel=Qwen3-8B, Setting=WAKVQ, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 56.08 | 88.48 | 80.59 | 73.2 | 66.69 | 73.01 | 96.57 | |
| ScaleSweepMSEModel=Qwen3-8B, Setting=WAKV, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 55.49 | 88.4 | 79.48 | 73.56 | 66.3 | 72.65 | 96.09 | |
| ScaleSweepModel=Qwen3-4B, Setting=WA, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 54.95 | 87.04 | 77.26 | 66.98 | 62.98 | 69.84 | 96.91 | |
| AbsMaxModel=Qwen3-8B, Setting=WAKV, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 54.9 | 88.55 | 83.92 | 72.97 | 67.4 | 73.55 | 97.29 | |
| ScaleSweepMSEModel=Qwen3-4B, Setting=WA, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 54.75 | 86.28 | 79.11 | 66.34 | 65.51 | 70.4 | 97.68 | |
| ScaleSweepModel=Qwen3-4B, Setting=WA, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 54.69 | 88.02 | 76.52 | 66.61 | 61.25 | 69.42 | 96.32 | |
| 4/6Model=Qwen3-4B, Setting=WA, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 54.21 | 86.73 | 78.56 | 66.02 | 63.85 | 69.87 | 96.95 | |
| 4/6Model=Qwen3-4B, Setting=WA, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 54.16 | 86.88 | 77.82 | 66.37 | 65.43 | 70.13 | 97.31 | |
| AbsMaxModel=Qwen3-4B, Setting=WA, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 53.74 | 85.82 | 77.82 | 66.48 | 64.09 | 69.59 | 96.56 | |
| 4/6Model=Qwen3-8B, Setting=WAKVQ, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 53.71 | 87.64 | 80.41 | 72.92 | 66.46 | 72.23 | 95.54 | |
| ScaleSweepMSEModel=Qwen3-8B, Setting=WAKVQ, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 53.55 | 88.48 | 80.22 | 73.18 | 65.98 | 72.28 | 95.61 | |
| ScaleSweepMSEModel=Qwen3-4B, Setting=WA, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 53.37 | 86.05 | 78.93 | 66.67 | 63.3 | 69.66 | 96.66 | |
| ScaleSweepModel=Qwen3-4B, Setting=WAKV, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 53.25 | 86.35 | 78.19 | 66.73 | 65.04 | 69.91 | 97.01 | |
| AbsMaxModel=Qwen3-8B, Setting=WAKVQ, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 52.98 | 87.11 | 82.62 | 71.83 | 67.48 | 72.4 | 95.77 | |
| AbsMaxModel=Qwen3-4B, Setting=WA, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 52.86 | 86.58 | 79.67 | 65.77 | 62.51 | 69.48 | 96.4 | |
| ScaleSweepModel=Qwen3-4B, Setting=WAKV, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 52.79 | 87.11 | 77.45 | 66.32 | 62.51 | 69.24 | 96.07 | |
| ScaleSweepMSEModel=Qwen3-4B, Setting=WAKV, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 52.35 | 84.99 | 79.48 | 65.65 | 63.3 | 69.16 | 95.96 | |
| 4/6Model=Qwen3-8B, Setting=WA, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 52.24 | 89.23 | 81.52 | 73.31 | 67.25 | 72.71 | 96.18 | |
| ScaleSweepModel=Qwen3-8B, Setting=WAKV, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 51.95 | 89.31 | 80.96 | 73.88 | 66.93 | 72.61 | 96.04 | |
| ScaleSweepModel=Qwen3-4B, Setting=WAKVQ, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 51.7 | 84 | 78.37 | 65.76 | 60.85 | 68.14 | 94.54 | |
| 4/6Model=Qwen3-4B, Setting=WAKV, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 51.67 | 85.06 | 80.22 | 65.68 | 62.27 | 68.98 | 95.72 | |
| ScaleSweepModel=Qwen3-4B, Setting=WAKVQ, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 51.53 | 83.7 | 76.34 | 65.91 | 64.96 | 68.49 | 95.03 | |
| 4/6Model=Qwen3-4B, Setting=WAKV, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 51.38 | 83.78 | 79.85 | 65.46 | 61.72 | 68.44 | 94.96 | |
| AbsMaxModel=Qwen3-8B, Setting=WAKVQ, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 51.19 | 88.32 | 82.07 | 72.58 | 67.56 | 72.34 | 95.69 | |
| AbsMaxModel=Qwen3-4B, Setting=WAKV, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 51.18 | 84.31 | 74.68 | 65.52 | 64.56 | 68.05 | 94.42 | |
| ScaleSweepMSEModel=Qwen3-4B, Setting=WAKV, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 51.02 | 85.37 | 78.19 | 65.64 | 62.12 | 68.47 | 95 | |
| ScaleSweepMSEModel=Qwen3-4B, Setting=WAKVQ, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 50.87 | 84.76 | 78.93 | 65.46 | 64.88 | 68.98 | 95.71 | |
| AbsMaxModel=Qwen3-4B, Setting=WAKV, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 49.33 | 84.23 | 76.52 | 65.12 | 62.43 | 67.53 | 93.7 | |
| ScaleSweepMSEModel=Qwen3-4B, Setting=WAKVQ, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 49.29 | 84.08 | 77.63 | 65.07 | 62.04 | 67.62 | 93.83 | |
| 4/6Model=Qwen3-4B, Setting=WAKVQ, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 49.17 | 81.88 | 76.71 | 65.23 | 62.51 | 67.1 | 93.1 | |
| 4/6Model=Qwen3-8B, Setting=WAKVQ, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 48.09 | 88.25 | 80.59 | 72.78 | 67.17 | 71.38 | 94.41 | |
| 4/6Model=Qwen3-8B, Setting=WAKV, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 47.91 | 88.55 | 81.33 | 73.04 | 67.25 | 71.62 | 94.73 | |
| AbsMaxModel=Qwen3-4B, Setting=WAKVQ, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 47.44 | 82.41 | 76.34 | 64.38 | 61.17 | 66.35 | 92.06 | |
| AbsMaxModel=Qwen3-4B, Setting=WAKVQ, Algorithm=GPTQ, Evaluation Mode=non-thinking2026.05 | 46.96 | 81.12 | 77.82 | 65.24 | 61.48 | 66.52 | 92.3 | |
| 4/6Model=Qwen3-4B, Setting=WAKVQ, Algorithm=RTN, Evaluation Mode=non-thinking2026.05 | 46.88 | 83.02 | 77.45 | 65.21 | 63.06 | 67.12 | 93.14 | |
| BF16Precision=BF162026.05 | 46.59 | 84.69 | 73.94 | 79.53 | 73.8 | 71.71 | 100 | |
| ScaleSweepSetting=WA, Method=RTN2026.05 | 43.09 | 79.68 | 70.79 | 78.27 | 72.45 | 68.86 | 96.03 | |
| ScaleSweepMSESetting=WA, Method=GPTQ2026.05 | 43.03 | 81.43 | 69.13 | 78.19 | 74.11 | 69.18 | 96.47 | |
| ScaleSweepMSESetting=WA, Method=RTN2026.05 | 42.77 | 82.03 | 73.2 | 78.17 | 73.09 | 69.85 | 97.41 | |
| ScaleSweepSetting=WA, Method=GPTQ2026.05 | 42.7 | 81.5 | 74.49 | 78.35 | 72.93 | 69.99 | 97.61 | |
| 4/6Setting=WA, Method=GPTQ2026.05 | 42.46 | 80.67 | 70.98 | 78.01 | 73.24 | 69.07 | 96.33 | |
| 4/6Setting=WA, Method=RTN2026.05 | 42.29 | 80.59 | 67.28 | 78.52 | 73.01 | 68.34 | 95.3 | |
| AbsMaxSetting=WA, Method=GPTQ2026.05 | 41.94 | 81.58 | 69.13 | 77.94 | 73.32 | 68.78 | 95.92 | |
| AbsMaxSetting=WA, Method=RTN2026.05 | 41.48 | 78.32 | 69.69 | 77.91 | 72.53 | 67.99 | 94.81 | |
| ScaleSweepSetting=WAKV, Method=RTN2026.05 | 41.29 | 79.91 | 69.69 | 78.12 | 72.53 | 68.31 | 95.26 | |
| ScaleSweepSetting=WAKV, Method=GPTQ2026.05 | 41.26 | 79.98 | 71.9 | 77.54 | 72.77 | 68.69 | 95.8 | |
| ScaleSweepMSESetting=WAKV, Method=RTN2026.05 | 41.06 | 78.85 | 70.24 | 78.09 | 72.85 | 68.22 | 95.13 | |
| ScaleSweepMSESetting=WAKV, Method=GPTQ2026.05 | 40.86 | 79.45 | 70.61 | 77.71 | 72.22 | 68.17 | 95.07 | |
| 4/6Setting=WAKV, Method=GPTQ2026.05 | 40.26 | 80.06 | 69.13 | 77.78 | 72.22 | 67.89 | 94.68 | |
| ScaleSweepSetting=WAKVQ, Method=RTN2026.05 | 39.72 | 77.33 | 69.13 | 77.43 | 71.27 | 66.98 | 93.4 | |
| ScaleSweepSetting=WAKVQ, Method=GPTQ2026.05 | 39.67 | 77.33 | 70.24 | 77.67 | 71.82 | 67.35 | 93.92 | |
| 4/6Setting=WAKV, Method=RTN2026.05 | 39.29 | 78.7 | 66.54 | 77.58 | 72.14 | 66.85 | 93.23 | |
| ScaleSweepMSESetting=WAKVQ, Method=GPTQ2026.05 | 38.6 | 78.24 | 68.58 | 77.42 | 72.06 | 66.98 | 93.41 | |
| AbsMaxSetting=WAKV, Method=GPTQ2026.05 | 38.4 | 77.94 | 70.61 | 77.95 | 71.35 | 67.25 | 93.78 | |
| AbsMaxSetting=WAKV, Method=RTN2026.05 | 38.21 | 76.95 | 68.95 | 77.31 | 71.35 | 66.55 | 92.81 | |
| ScaleSweepMSESetting=WAKVQ, Method=RTN2026.05 | 37.86 | 74.98 | 69.32 | 77.38 | 71.59 | 66.23 | 92.35 | |
| 4/6Setting=WAKVQ, Method=RTN2026.05 | 36.69 | 74.22 | 68.02 | 77.43 | 72.14 | 65.7 | 91.62 | |
| 4/6Setting=WAKVQ, Method=GPTQ2026.05 | 36.45 | 76.57 | 67.65 | 77.62 | 72.14 | 66.09 | 92.16 | |
| AbsMaxSetting=WAKVQ, Method=GPTQ2026.05 | 34.53 | 73.16 | 68.21 | 76.92 | 71.51 | 64.87 | 90.46 | |
| AbsMaxSetting=WAKVQ, Method=RTN2026.05 | 34.04 | 69.9 | 70.61 | 76.99 | 70.88 | 64.48 | 89.93 |