Logical Reasoning on LSAT (Statistical Significance)
0.038Mean ImprovementiPASwo
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| iPASwoModel=DeepSeek-R1-Distill-8B2025.09 | 0.038 | 0.018 | 0 | |
| iPASaModel=DeepSeek-R1-Distill-8B2025.09 | 0.036 | 0.011 | 0 | |
| PASfModel=Llama-3.1-8B-Instruct2025.09 | 0.03 | 0.01 | 0 | |
| PASfModel=DeepSeek-R1-Distill-8B2025.09 | 0.026 | 0.008 | 0 | |
| iPASaModel=Llama-3.1-8B-Instruct2025.09 | 0.02 | 0 | 0.02 | |
| iPASwoModel=Llama-3.1-8B-Instruct2025.09 | 0.01 | -0.01 | 0.1 | |
| PASfModel=Nous-Hermes-2-Mistral-7B-DPO2025.09 | 0.009 | -0.002 | 0.05 | |
| iPASaModel=Nous-Hermes-2-Mistral-7B-DPO2025.09 | 0.007 | 0 | 0.03 | |
| iPASwoModel=Nous-Hermes-2-Mistral-7B-DPO2025.09 | 0.004 | -0.008 | 0.24 |