Natural Language Understanding on LM Evaluation Harness (MMLU-CoT, GSM8K)
72.76MMLU (CoT)BF16 Baseline
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BF16 BaselineFormat=BF16, Model=Llama-3.1-8B-Instruct2025.11 | 72.76 | 85.06 | 80.01 | 77.9 | 78.93 | 100 | |
| GPTQ-WUSHFormat=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 69.69 | 80.11 | 78.52 | 76.09 | 76.1 | 96.4 | |
| GPTQ-H (MR-GPTQ)Format=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 69.12 | 80.8 | 78.17 | 75.24 | 75.84 | 96.08 | |
| SmoothQuantFormat=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 68.9 | 79.5 | 79.5 | 74.7 | 75.7 | 95.9 | |
| GPTQ-IFormat=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 68.85 | 82.6 | 78.26 | 74.51 | 75.72 | 95.92 | |
| RTN-WUSHFormat=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 68.83 | 78.57 | 78.22 | 75.47 | 75.28 | 95.37 | |
| RTN-IFormat=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 68.26 | 78.39 | 78.15 | 74.11 | 74.73 | 94.67 | |
| GPTQ-WUSHFormat=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 67.79 | 77.41 | 77.44 | 74.78 | 74.35 | 94.2 | |
| RTN-HFormat=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 67.41 | 78.01 | 77.31 | 73.48 | 74.05 | 93.82 | |
| GPTQ-H (MR-GPTQ)Format=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 67.19 | 75.7 | 76.91 | 74.8 | 73.65 | 93.31 | |
| RTN-WUSHFormat=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 66.85 | 75.16 | 77.28 | 73.56 | 73.21 | 92.75 | |
| QuaRotFormat=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 66.5 | 77.4 | 77.25 | 75.14 | 74.1 | 93.8 | |
| SpinQuantFormat=NVFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 66.5 | 76.1 | 76.96 | 75.32 | 73.7 | 93.4 | |
| SmoothQuantFormat=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 63.93 | 68.54 | 75.1 | 73.56 | 70.3 | 89.06 | |
| GPTQ-IFormat=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 63.49 | 68.46 | 76.01 | 74.51 | 70.62 | 89.47 | |
| RTN-HFormat=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 62.38 | 72.48 | 75.29 | 71.67 | 70.45 | 89.26 | |
| RTN-IFormat=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 62.21 | 67.85 | 73.99 | 73.24 | 69.32 | 87.83 | |
| SpinQuantFormat=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 61.8 | 68.16 | 74.87 | 72.93 | 69.4 | 88 | |
| QuaRotFormat=MXFP4, Quantization=W4A4, Model=Llama-3.1-8B-Instruct2025.11 | 49.86 | 56.94 | 73.5 | 71.43 | 62.9 | 79.7 |