List your company
ResearchFormerly WizWand

Multi-subject Knowledge Reasoning on MMLU-Pro

73.44Pass@1FP16
49.57255.768561.96568.1615May 9, 2026
Updated 5mo ago

Evaluation results

MethodLinks
FP16Model=Qwen3-8B, Calibration Dataset=N/A, Quantization Bit=N/A, Group Size=N/A2026.0573.44
GPTQModel=Qwen3-8B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.0572.67
ReasoningQATModel=Qwen3-8B, Calibration Dataset=Mixed, Quantization Bit=4, Group Size=1282026.0572.65
ParoQuant++Model=Qwen3-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0572.59
ParoQuantModel=Qwen3-8B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.0572.43
GPTQ++Model=Qwen3-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0572.33
LAQuantModel=Qwen3-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0571.52
FP16Model=Qwen3-4B, Calibration Dataset=N/A, Quantization Bit=N/A, Group Size=N/A2026.0570.23
ParoQuantModel=Qwen3-4B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.0569.41
ParoQuant++Model=Qwen3-4B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0568.73
LAQuantModel=Qwen3-4B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0568.41
ReasoningQATModel=Qwen3-4B, Calibration Dataset=Mixed, Quantization Bit=4, Group Size=1282026.0568.28
GPTQModel=Qwen3-4B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.0568.03
GPTQ++Model=Qwen3-4B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0568
FP16Model=R1-Distill-Llama-8B, Calibration Dataset=N/A, Quantization Bit=N/A, Group Size=N/A2026.0558.84
FP16Model=Qwen3-1.7B, Calibration Dataset=N/A, Quantization Bit=N/A, Group Size=N/A2026.0557.31
GPTQModel=R1-Distill-Llama-8B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.0556.78
ParoQuant++Model=R1-Distill-Llama-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0556.77
ParoQuantModel=R1-Distill-Llama-8B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.0556.69
GPTQ++Model=R1-Distill-Llama-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0556.3
ReasoningQATModel=R1-Distill-Llama-8B, Calibration Dataset=Mixed, Quantization Bit=4, Group Size=1282026.0555.96
LAQuantModel=Qwen3-1.7B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0555.58
ParoQuant++Model=Qwen3-1.7B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0555.44
LAQuantModel=R1-Distill-Llama-8B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0554.98
ParoQuantModel=Qwen3-1.7B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.0554.58
ReasoningQATModel=Qwen3-1.7B, Calibration Dataset=Mixed, Quantization Bit=4, Group Size=1282026.0554.44
GPTQ++Model=Qwen3-1.7B, Calibration Dataset=OpenR1, Quantization Bit=4, Group Size=1282026.0554.21
GPTQModel=Qwen3-1.7B, Calibration Dataset=Generic, Quantization Bit=4, Group Size=1282026.0550.49