Logical Reasoning on BBH (Pass@N)
58.4Pass@1CaSP
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| CaSPModel=Qwen2.5-7B2025.10 | 58.4 | 69.2 | 77.3 | 83 | 86.8 | 89.3 | 91 | 92.1 | |
| PSRModel=Qwen2.5-7B2025.10 | 54.9 | 62.7 | 68.8 | 73.4 | 76.8 | 79.4 | 81.4 | 82.8 | |
| GRPOModel=Qwen2.5-7B2025.10 | 54.4 | 62.8 | 69.8 | 75.6 | 79.3 | 82.5 | 84.9 | 86.8 | |
| Base ModelModel=Qwen2.5-7B2025.10 | 42.4 | 59.3 | 74.4 | 84.6 | 89.9 | 92.4 | 93.6 | 94.2 | |
| W-REINFORCEModel=Qwen2.5-7B2025.10 | 15.4 | 21.9 | 27.8 | 32.3 | 35.4 | 37.2 | 38.3 | 38.8 |