List your company
ResearchFormerly WizWand

Reasoning on MMLU-Pro (Pass@1)

73.44Pass@1FP16
49.57255.768561.96568.1615May 9, 2026
Updated 5mo ago

Evaluation results

MethodLinks
FP16Model=Qwen3-8B, Speedup=1.0×2026.0573.44
GPTQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.0572.67
R-QATModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.0572.65
ParoQ++Model=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.0572.59
ParoQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.0572.43
LAQuantModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.0571.52
FP16Model=Qwen3-4B, Speedup=1.0×2026.0570.23
ParoQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.0569.41
ParoQ++Model=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.0568.73
LAQuantModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.0568.41
R-QATModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.0568.28
GPTQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.0568.03
FP16Model=R1-Distill-Llama-8B, Speedup=1.0×2026.0558.84
FP16Model=Qwen3-1.7B, Speedup=1.0×2026.0557.31
GPTQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.0556.78
ParoQ++Model=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.0556.77
ParoQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.0556.69
R-QATModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.0555.96
LAQuantModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.0555.58
ParoQ++Model=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.0555.44
LAQuantModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.0554.98
ParoQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.0554.58
R-QATModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.0554.44
GPTQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.0550.49