Multi-subject Knowledge Reasoning on MMLU-Pro
73.44Pass@1FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Model=Qwen3-8B, Calibration Dataset=N/A, Quantization Bit=N/A, Group Size=N/A2026.05 | 73.44 | |
| GPTQModel=Qwen3-8B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.05 | 72.67 | |
| ReasoningQATModel=Qwen3-8B, Calibration Dataset=Mixed, Quantization Bit=4, Group Size=1282026.05 | 72.65 | |
| ParoQuant++Model=Qwen3-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 72.59 | |
| ParoQuantModel=Qwen3-8B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.05 | 72.43 | |
| GPTQ++Model=Qwen3-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 72.33 | |
| LAQuantModel=Qwen3-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 71.52 | |
| FP16Model=Qwen3-4B, Calibration Dataset=N/A, Quantization Bit=N/A, Group Size=N/A2026.05 | 70.23 | |
| ParoQuantModel=Qwen3-4B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.05 | 69.41 | |
| ParoQuant++Model=Qwen3-4B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 68.73 | |
| LAQuantModel=Qwen3-4B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 68.41 | |
| ReasoningQATModel=Qwen3-4B, Calibration Dataset=Mixed, Quantization Bit=4, Group Size=1282026.05 | 68.28 | |
| GPTQModel=Qwen3-4B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.05 | 68.03 | |
| GPTQ++Model=Qwen3-4B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 68 | |
| FP16Model=R1-Distill-Llama-8B, Calibration Dataset=N/A, Quantization Bit=N/A, Group Size=N/A2026.05 | 58.84 | |
| FP16Model=Qwen3-1.7B, Calibration Dataset=N/A, Quantization Bit=N/A, Group Size=N/A2026.05 | 57.31 | |
| GPTQModel=R1-Distill-Llama-8B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.05 | 56.78 | |
| ParoQuant++Model=R1-Distill-Llama-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 56.77 | |
| ParoQuantModel=R1-Distill-Llama-8B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.05 | 56.69 | |
| GPTQ++Model=R1-Distill-Llama-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 56.3 | |
| ReasoningQATModel=R1-Distill-Llama-8B, Calibration Dataset=Mixed, Quantization Bit=4, Group Size=1282026.05 | 55.96 | |
| LAQuantModel=Qwen3-1.7B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 55.58 | |
| ParoQuant++Model=Qwen3-1.7B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 55.44 | |
| LAQuantModel=R1-Distill-Llama-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 54.98 | |
| ParoQuantModel=Qwen3-1.7B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.05 | 54.58 | |
| ReasoningQATModel=Qwen3-1.7B, Calibration Dataset=Mixed, Quantization Bit=4, Group Size=1282026.05 | 54.44 | |
| GPTQ++Model=Qwen3-1.7B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.05 | 54.21 | |
| GPTQModel=Qwen3-1.7B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.05 | 50.49 |