Reasoning on LSAT (Pass@1)
85.57Pass@1FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Model=Qwen3-8B, Speedup=1.0×2026.05 | 85.57 | |
| FP16Model=Qwen3-8B, Quantization Bits=FP162026.05 | 85.57 | |
| R-QATModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 85.24 | |
| LAQuantModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 85.24 | |
| ParoQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.05 | 84.43 | |
| ParoQ++Model=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.05 | 83.99 | |
| LAQuantModel=Qwen3-8B, Quantization Bits=3-bit2026.05 | 83.8 | |
| GPTQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 82.31 | |
| FP16Model=Qwen3-4B, Speedup=1.0×2026.05 | 81.14 | |
| FP16Model=Qwen3-4B, Quantization Bits=FP162026.05 | 81.14 | |
| LAQuantModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 81.03 | |
| ParoQ++Model=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.05 | 80.87 | |
| ParoQ++Model=Qwen3-8B, Quantization Bits=3-bit2026.05 | 79.48 | |
| ParoQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.05 | 79.43 | |
| ParoQModel=Qwen3-8B, Quantization Bits=3-bit2026.05 | 78.53 | |
| R-QATModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 78.45 | |
| GPTQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 78.1 | |
| R-QATModel=Qwen3-8B, Quantization Bits=3-bit2026.05 | 77.23 | |
| LAQuantModel=Qwen3-4B, Quantization Bits=3-bit2026.05 | 77.15 | |
| ParoQ++Model=Qwen3-4B, Quantization Bits=3-bit2026.05 | 77.07 | |
| GPTQModel=Qwen3-8B, Quantization Bits=3-bit2026.05 | 69.92 | |
| ParoQModel=Qwen3-4B, Quantization Bits=3-bit2026.05 | 68.94 | |
| R-QATModel=Qwen3-4B, Quantization Bits=3-bit2026.05 | 67.64 | |
| FP16Model=Qwen3-1.7B, Speedup=1.0×2026.05 | 64.1 | |
| FP16Model=Qwen3-1.7B, Quantization Bits=FP162026.05 | 64.1 | |
| LAQuantModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 61.14 | |
| ParoQ++Model=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.05 | 60.52 | |
| FP16Model=R1-Distill-Llama-8B, Speedup=1.0×2026.05 | 59.78 | |
| FP16Model=R1-Distill-Llama-8B, Quantization Bits=FP162026.05 | 59.78 | |
| LAQuantModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 58.67 | |
| ParoQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.05 | 58.3 | |
| R-QATModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 57.01 | |
| GPTQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 56.66 | |
| ParoQ++Model=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.05 | 56.66 | |
| LAQuantModel=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 54.86 | |
| ParoQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.05 | 54.81 | |
| R-QATModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 53.48 | |
| LAQuantModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 50.98 | |
| ParoQ++Model=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 49.3 | |
| ParoQ++Model=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 48.89 | |
| R-QATModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 48.48 | |
| GPTQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 48.29 | |
| R-QATModel=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 47.69 | |
| GPTQModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 42.47 | |
| GPTQModel=Qwen3-4B, Quantization Bits=3-bit2026.05 | 40.41 | |
| ParoQModel=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 36.1 | |
| ParoQModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 29.65 | |
| GPTQModel=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 18.78 |