Logical Reasoning on Logical Deduction
81.03Pass@1MAPR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MAPRBase Model=Qwen3-14B-Base2025.09 | 81.03 | — | |
| GRPOBase Model=Qwen3-14B-Base2025.09 | 80.81 | — | |
| HtTMethod Category=PRIOR RBRS2025.06 | 1 | — | |
| RGFBMethod Category=PRIOR RBRS2025.06 | 1 | — | |
| Chain-of-LogicMethod Category=PRIOR RBRS2025.06 | 1 | — | |
| OpenAI o3-miniMethod Category=FRONTIER REASONERS2025.06 | 1 | — | |
| DeepSeek-R1Method Category=FRONTIER REASONERS2025.06 | 0.983 | — | |
| RULEREASONER-8BMethod Category=RULEREASONER (Ours), Model Scale=8B2025.06 | 0.983 | 0.4 | |
| Claude-3.7-SonnetMethod Category=FRONTIER REASONERS2025.06 | 0.97 | — | |
| ADARFTMethod Category=CURRICULUM LEARNING2025.06 | 0.966 | — | |
| RULEREASONER-4BMethod Category=RULEREASONER (Ours), Model Scale=4B2025.06 | 0.963 | 0.2 | |
| Easy-to-hard RLMethod Category=CURRICULUM LEARNING2025.06 | 0.96 | — | |
| DAPOMethod Category=ADVANCED RLVRS2025.06 | 0.953 | — | |
| Data-balance RLMethod Category=CURRICULUM LEARNING2025.06 | 0.953 | — | |
| GRPOMethod Category=ADVANCED RLVRS2025.06 | 0.903 | — | |
| OpenAI o1Method Category=FRONTIER REASONERS2025.06 | 0.88 | — | |
| SFT w/ Short CoTMethod Category=BEHAVIORAL CLONING, Reasoning Strategy=Short CoT2025.06 | 0.876 | — | |
| SFT w/o CoTMethod Category=BEHAVIORAL CLONING, Reasoning Strategy=w/o CoT2025.06 | 0.859 | — | |
| Dr. GRPOMethod Category=ADVANCED RLVRS2025.06 | 0.843 | — | |
| SFT w/ Long CoTMethod Category=BEHAVIORAL CLONING, Reasoning Strategy=Long CoT2025.06 | 0.796 | — |