Logical Reasoning on Knight-and-Knaves
81.8Pass@1 Rate (4-8 Roles, ID)RLCSD
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| RLCSDModel=Qwen3-8B2026.06 | 81.8 | 79 | 70 | 65 | 74 | |
| RLCSDModel=Qwen3-4B2026.06 | 78.6 | 73 | 66 | 50 | 66.9 | |
| RLSDModel=Qwen3-4B2026.06 | 76.8 | 72 | 63 | 48 | 65 | |
| GRPOModel=Qwen3-8B2026.06 | 76.8 | 75 | 63 | 49 | 66 | |
| RLSDModel=Qwen3-8B2026.06 | 76.6 | 77 | 64 | 52 | 67.4 | |
| GRPOModel=Qwen3-4B2026.06 | 75.4 | 71 | 61 | 45 | 63.1 | |
| RLCSDModel=Olmo-3-7B2026.06 | 75.4 | 76 | 65 | 48 | 66.1 | |
| OPSDModel=Qwen3-8B2026.06 | 75.2 | 74 | 61 | 49 | 64.8 | |
| SRPOModel=Qwen3-4B2026.06 | 75 | 71 | 61 | 45 | 63 | |
| SRPOModel=Qwen3-8B2026.06 | 74.8 | 72 | 60 | 49 | 64 | |
| SDPOModel=Qwen3-4B2026.06 | 74.4 | 72 | 62 | 45 | 63.4 | |
| GRPOModel=Olmo-3-7B2026.06 | 73.8 | 69 | 63 | 39 | 61.2 | |
| RLSDModel=Olmo-3-7B2026.06 | 73.8 | 65 | 61 | 38 | 59.4 | |
| OPSDModel=Qwen3-4B2026.06 | 73.4 | 71 | 62 | 42 | 62.1 | |
| BaseModel=Qwen3-4B2026.06 | 73.2 | 67 | 58 | 42 | 60.1 | |
| SDPOModel=Olmo-3-7B2026.06 | 73.2 | 67 | 59 | 46 | 61.3 | |
| SRPOModel=Olmo-3-7B2026.06 | 73.2 | 68 | 61 | 40 | 60.6 | |
| BaseModel=Qwen3-8B2026.06 | 72.4 | 67 | 55 | 44 | 59.6 | |
| SDPOModel=Qwen3-8B2026.06 | 72.4 | 72 | 56 | 46 | 61.6 | |
| OPSDModel=Olmo-3-7B2026.06 | 72.4 | 69 | 62 | 38 | 60.4 | |
| BaseModel=Olmo-3-7B2026.06 | 70.6 | 64 | 55 | 35 | 56.2 | |
| RLCSDModel=Qwen3-1.7B2026.06 | 70 | 63 | 63 | 38 | 58.5 | |
| GRPOModel=Qwen3-1.7B2026.06 | 67.4 | 59 | 52 | 34 | 53.1 | |
| SDPOModel=Qwen3-1.7B2026.06 | 67.4 | 61 | 54 | 30 | 53.1 | |
| RLSDModel=Qwen3-1.7B2026.06 | 66.8 | 59 | 50 | 35 | 52.7 | |
| OPSDModel=Qwen3-1.7B2026.06 | 64.4 | 55 | 52 | 32 | 50.9 | |
| SRPOModel=Qwen3-1.7B2026.06 | 64.4 | 58 | 47 | 33 | 50.6 | |
| BaseModel=Qwen3-1.7B2026.06 | 63.2 | 53 | 43 | 31 | 47.6 |