NLU and Math Reasoning on MMLU, GSM8k, HellaSwag, WinoGrande (test)
77.18MMLU AccuracyFP16
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FP16Format=FP162025.09 | 77.18 | 91.96 | 79.84 | 74.27 | 80.81 | — | |
| RTNFormat=NVFP, Quantization=RTN2025.09 | 75.73 | 91.28 | 78.36 | 73.16 | 79.63 | 98.54 | |
| GPTQ+Had16Format=NVFP, Quantization=GPTQ+Had162025.09 | 75.49 | 91.43 | 78.38 | 74.51 | 79.95 | 98.94 | |
| GPTQ+Had128Format=NVFP, Quantization=GPTQ+Had1282025.09 | 75.1 | 90.52 | 78.3 | 72.77 | 79.17 | 97.97 | |
| RTN+Had16Format=NVFP, Quantization=RTN+Had162025.09 | 74.98 | 92.04 | 77.76 | 72.38 | 79.29 | 98.12 | |
| GPTQFormat=NVFP, Quantization=GPTQ2025.09 | 74.88 | 91.28 | 78.4 | 74.51 | 79.77 | 98.71 | |
| RTN+Had128Format=NVFP, Quantization=RTN+Had1282025.09 | 74.46 | 91.13 | 77.6 | 71.98 | 78.79 | 97.5 | |
| GPTQ+Had32Format=MXFP, Quantization=GPTQ+Had322025.09 | 74.36 | 89.92 | 77.64 | 72.53 | 78.61 | 97.28 | |
| GPTQ+Had128Format=MXFP, Quantization=GPTQ+Had1282025.09 | 74.11 | 89.92 | 77.77 | 71.11 | 78.23 | 96.8 | |
| RTN+Had32Format=MXFP, Quantization=RTN+Had322025.09 | 73.19 | 89.54 | 75.95 | 71.67 | 77.59 | 96.01 | |
| RTN+Had128Format=MXFP, Quantization=RTN+Had1282025.09 | 73.17 | 85.6 | 76.8 | 72.14 | 76.93 | 95.19 | |
| RTNFormat=MXFP, Quantization=RTN2025.09 | 72.92 | 90.22 | 76.68 | 71.51 | 77.83 | 96.31 | |
| GPTQFormat=MXFP, Quantization=GPTQ2025.09 | 72.57 | 89.54 | 76.5 | 72.45 | 77.77 | 96.23 |