Zero-shot Language Modeling and Reasoning on MMLU-CoT, GSM8k, HellaSwag, and WinoGrande
72.76MMLU-CoT AccuracyFP16
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FP16Format=Baseline2025.09 | 72.76 | 85.06 | 80.01 | 77.9 | 78.93 | 100 | |
| RTNFormat=INT82025.09 | 72.5 | 84.8 | 80.2 | 77.4 | 78.73 | 99.74 | |
| GPTQFormat=INT82025.09 | 72.4 | 84.4 | 80 | 77.3 | 78.53 | 99.48 | |
| RTNFormat=FP82025.09 | 72.4 | 84.7 | 79.8 | 77.7 | 78.65 | 99.64 | |
| GPTQFormat=FP82025.09 | 71.8 | 84.5 | 79.9 | 78.1 | 78.58 | 99.55 | |
| MR-GPTQFormat=NVINT42025.09 | 69.71 | 82.26 | 79.14 | 75.53 | 76.66 | 97.12 | |
| MR-GPTQFormat=NVFP42025.09 | 69.12 | 80.8 | 78.17 | 75.24 | 75.84 | 96.08 | |
| SmoothQuantFormat=NVFP42025.09 | 68.9 | 79.5 | 79.5 | 74.7 | 75.7 | 95.9 | |
| GPTQFormat=NVFP42025.09 | 68.85 | 82.6 | 78.26 | 74.51 | 75.72 | 95.92 | |
| GPTQFormat=NVINT42025.09 | 68.69 | 81.58 | 77.59 | 73.4 | 75.32 | 95.42 | |
| RTN + HTFormat=NVINT42025.09 | 68.59 | 81.73 | 78.38 | 74.27 | 75.74 | 95.96 | |
| RTNFormat=NVINT42025.09 | 68.56 | 78.17 | 78.64 | 75.14 | 75.13 | 95.18 | |
| RTN+HTFormat=INT42025.09 | 68.3 | 79.61 | 77.6 | 73.48 | 74.75 | 94.71 | |
| RTNFormat=NVFP42025.09 | 68.26 | 78.39 | 78.15 | 74.11 | 74.73 | 94.67 | |
| RTN + HTFormat=NVFP42025.09 | 67.41 | 78.01 | 77.31 | 73.48 | 74.05 | 93.82 | |
| MR-GPTQFormat=MXFP42025.09 | 67.19 | 75.7 | 76.91 | 74.8 | 73.65 | 93.31 | |
| QuaRotFormat=NVFP42025.09 | 66.5 | 77.4 | 77.25 | 75.14 | 74.1 | 93.8 | |
| SpinQuantFormat=NVFP42025.09 | 66.5 | 76.1 | 76.96 | 75.32 | 73.7 | 93.4 | |
| GPTQFormat=INT42025.09 | 66.36 | 76.65 | 77.38 | 72.48 | 73.21 | 92.75 | |
| RTNFormat=INT42025.09 | 65.96 | 74.68 | 77.62 | 74.19 | 73.11 | 92.63 | |
| MR-GPTQFormat=MXINT42025.09 | 65.48 | 74.83 | 76.63 | 73.09 | 72.51 | 91.86 | |
| SmoothQuantFormat=MXFP42025.09 | 63.93 | 68.54 | 75.1 | 73.56 | 70.3 | 89.06 | |
| GPTQFormat=MXFP42025.09 | 63.49 | 68.46 | 76.01 | 74.51 | 70.62 | 89.47 | |
| RTN + HTFormat=MXFP42025.09 | 62.38 | 72.48 | 75.29 | 71.67 | 70.45 | 89.26 | |
| RTNFormat=MXFP42025.09 | 62.21 | 67.85 | 73.99 | 73.24 | 69.32 | 87.83 | |
| SpinQuantFormat=MXFP42025.09 | 61.8 | 68.16 | 74.87 | 72.93 | 69.4 | 88 | |
| GPTQFormat=MXINT42025.09 | 61.22 | 67.7 | 75.04 | 71.67 | 68.91 | 87.3 | |
| RTN + HTFormat=MXINT42025.09 | 58.44 | 61.64 | 73.94 | 71.19 | 66.3 | 84 | |
| RTNFormat=MXINT42025.09 | 55.06 | 56.79 | 72.06 | 68.27 | 63.05 | 79.87 | |
| QuaRotFormat=MXFP42025.09 | 49.86 | 56.94 | 73.5 | 71.43 | 62.9 | 79.7 |