List your company
ResearchFormerly WizWand

Reasoning on LSAT (Pass@1)

85.57Pass@1FP16
16.108434.141752.17570.2083May 9, 2026
Updated 5mo ago

Evaluation results

MethodLinks
FP16Model=Qwen3-8B, Speedup=1.0×2026.0585.57
FP16Model=Qwen3-8B, Quantization Bits=FP162026.0585.57
R-QATModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.0585.24
LAQuantModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.0585.24
ParoQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.0584.43
ParoQ++Model=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.0583.99
LAQuantModel=Qwen3-8B, Quantization Bits=3-bit2026.0583.8
GPTQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.0582.31
FP16Model=Qwen3-4B, Speedup=1.0×2026.0581.14
FP16Model=Qwen3-4B, Quantization Bits=FP162026.0581.14
LAQuantModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.0581.03
ParoQ++Model=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.0580.87
ParoQ++Model=Qwen3-8B, Quantization Bits=3-bit2026.0579.48
ParoQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.0579.43
ParoQModel=Qwen3-8B, Quantization Bits=3-bit2026.0578.53
R-QATModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.0578.45
GPTQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.0578.1
R-QATModel=Qwen3-8B, Quantization Bits=3-bit2026.0577.23
LAQuantModel=Qwen3-4B, Quantization Bits=3-bit2026.0577.15
ParoQ++Model=Qwen3-4B, Quantization Bits=3-bit2026.0577.07
GPTQModel=Qwen3-8B, Quantization Bits=3-bit2026.0569.92
ParoQModel=Qwen3-4B, Quantization Bits=3-bit2026.0568.94
R-QATModel=Qwen3-4B, Quantization Bits=3-bit2026.0567.64
FP16Model=Qwen3-1.7B, Speedup=1.0×2026.0564.1
FP16Model=Qwen3-1.7B, Quantization Bits=FP162026.0564.1
LAQuantModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.0561.14
ParoQ++Model=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.0560.52
FP16Model=R1-Distill-Llama-8B, Speedup=1.0×2026.0559.78
FP16Model=R1-Distill-Llama-8B, Quantization Bits=FP162026.0559.78
LAQuantModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.0558.67
ParoQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.0558.3
R-QATModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.0557.01
GPTQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.0556.66
ParoQ++Model=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.0556.66
LAQuantModel=Qwen3-1.7B, Quantization Bits=3-bit2026.0554.86
ParoQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.0554.81
R-QATModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.0553.48
LAQuantModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.0550.98
ParoQ++Model=Qwen3-1.7B, Quantization Bits=3-bit2026.0549.3
ParoQ++Model=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.0548.89
R-QATModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.0548.48
GPTQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.0548.29
R-QATModel=Qwen3-1.7B, Quantization Bits=3-bit2026.0547.69
GPTQModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.0542.47
GPTQModel=Qwen3-4B, Quantization Bits=3-bit2026.0540.41
ParoQModel=Qwen3-1.7B, Quantization Bits=3-bit2026.0536.1
ParoQModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.0529.65
GPTQModel=Qwen3-1.7B, Quantization Bits=3-bit2026.0518.78