Reasoning on MMLU-Pro (Pass@1)
73.44Pass@1FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Model=Qwen3-8B, Speedup=1.0×2026.05 | 73.44 | |
| GPTQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 72.67 | |
| R-QATModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 72.65 | |
| ParoQ++Model=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.05 | 72.59 | |
| ParoQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.05 | 72.43 | |
| LAQuantModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 71.52 | |
| FP16Model=Qwen3-4B, Speedup=1.0×2026.05 | 70.23 | |
| ParoQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.05 | 69.41 | |
| ParoQ++Model=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.05 | 68.73 | |
| LAQuantModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 68.41 | |
| R-QATModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 68.28 | |
| GPTQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 68.03 | |
| FP16Model=R1-Distill-Llama-8B, Speedup=1.0×2026.05 | 58.84 | |
| FP16Model=Qwen3-1.7B, Speedup=1.0×2026.05 | 57.31 | |
| GPTQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 56.78 | |
| ParoQ++Model=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.05 | 56.77 | |
| ParoQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.05 | 56.69 | |
| R-QATModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 55.96 | |
| LAQuantModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 55.58 | |
| ParoQ++Model=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.05 | 55.44 | |
| LAQuantModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 54.98 | |
| ParoQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.05 | 54.58 | |
| R-QATModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 54.44 | |
| GPTQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 50.49 |