Mathematical Reasoning on MATH500 (Pass@1)
96.4Pass@1No Defense
Evaluation Results
| Method | Links | |
|---|---|---|
| No DefenseBackbone=R1-Qwen-14B2025.08 | 96.4 | |
| No DefenseBackbone=R1-Qwen-32B2025.08 | 95.5 | |
| SGPOBackbone=QwQ-32B, Judge Model=DeepSeek-V3-03242025.05 | 95 | |
| BaselineBackbone=QwQ-32B2025.05 | 94.6 | |
| GRPOBackbone=QwQ-32B2025.05 | 94.6 | |
| ReasoningGuardBackbone=R1-Qwen-14B2025.08 | 94.5 | |
| SAFEPATH-ZSBackbone=R1-Qwen-32B2025.08 | 94.5 | |
| ReasoningGuardBackbone=R1-Qwen-32B2025.08 | 93.6 | |
| ThinkingIBackbone=R1-Qwen-32B2025.08 | 93.5 | |
| SafeKeyBackbone=R1-Qwen-14B2025.08 | 92.7 | |
| ThinkingIBackbone=R1-Qwen-14B2025.08 | 92.7 | |
| SGPOBackbone=DeepSeek-R1-Distill-Qwen-7B, Judge Model=Qwen3-235B-A22B2025.05 | 92 | |
| SGPOBackbone=DeepSeek-R1-Distill-Qwen-7B, Judge Model=QwQ-32B2025.05 | 92 | |
| ParaphraseBackbone=R1-Qwen-14B2025.08 | 91.8 | |
| SGPOBackbone=DeepSeek-R1-Distill-Qwen-7B, Judge Model=DeepSeek-V3-03242025.05 | 91 | |
| ParaphraseBackbone=R1-Qwen-32B2025.08 | 90.9 | |
| SGPOBackbone=DeepSeek-R1-Distill-Qwen-7B, Judge Model=o4-mini-04162025.05 | 90.8 | |
| SGPOBackbone=DeepSeek-R1-Distill-Qwen-7B, Judge Model=QwQ-32B, Stability Parameters=w/o {beta,gamma}2025.05 | 90.2 | |
| GRPOBackbone=DeepSeek-R1-Distill-Qwen-7B2025.05 | 89.4 | |
| SAFEPATH-ZSBackbone=R1-Qwen-14B2025.08 | 89.1 | |
| MinPROModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 88.1 | |
| M2POModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 87.7 | |
| CISPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 87.4 | |
| BaselineBackbone=DeepSeek-R1-Distill-Qwen-7B2025.05 | 87 | |
| MinPROModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 86.5 | |
| M2POModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 85.8 | |
| GSPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 85.4 | |
| CISPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 84.5 | |
| GRPOBackbone=DeepSeek-R1-Distill-Llama-8B2025.05 | 84 | |
| SGPOBackbone=DeepSeek-R1-Distill-Llama-8B, Judge Model=Claude-3.72025.05 | 83.6 | |
| BaselineBackbone=Qwen2.5-32B-Instruct2025.05 | 83.6 | |
| GRPOBackbone=Qwen2.5-32B-Instruct2025.05 | 83.2 | |
| SGPOBackbone=Qwen2.5-32B-Instruct, Judge Model=o4-mini-04162025.05 | 83 | |
| SGPOBackbone=Qwen2.5-14B-Instruct, Judge Model=o4-mini-04162025.05 | 81 | |
| SetPO+GRPOBase Model=Qwen2.5-Math-7B, Optimization Algorithm=GRPO2026.02 | 80.8 | |
| BaselineBackbone=Qwen2.5-14B-Instruct2025.05 | 80.4 | |
| GRPOBackbone=Qwen2.5-14B-Instruct2025.05 | 80.2 | |
| GRPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 80 | |
| SCR (Ours)Backbone=Qwen2.5-7B-Instruct2026.01 | 80 | |
| SetPO+DAPOBase Model=Qwen2.5-Math-7B, Optimization Algorithm=DAPO2026.02 | 79.2 | |
| RealSafe-R1Backbone=R1-Qwen-32B2025.08 | 79.1 | |
| SFT + GRPOBackbone=Qwen2.5-7B-Instruct2026.01 | 78.8 | |
| GSPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 78.4 | |
| DAPOBase Model=Qwen2.5-Math-7B2026.02 | 77.6 | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.01 | 77.4 | |
| BaselineBackbone=DeepSeek-R1-Distill-Llama-8B2025.05 | 77.4 | |
| SCR-Stage IBackbone=Qwen2.5-7B-Instruct2026.01 | 77 | |
| R1-zero-DivBase Model=Qwen2.5-Math-7B2026.02 | 76.9 | |
| RealSafe-R1Backbone=R1-Qwen-14B2025.08 | 76.4 | |
| SetPO+GSPOBase Model=Qwen2.5-Math-7B, Optimization Algorithm=GSPO2026.02 | 75.7 | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.01 | 75.6 | |
| SCR-SFTBackbone=Qwen2.5-7B-Instruct2026.01 | 74.2 | |
| GRPOBase Model=Qwen2.5-Math-7B2026.02 | 73.5 | |
| GSPOBase Model=Qwen2.5-Math-7B2026.02 | 73 | |
| SCR (Ours)Backbone=Qwen2.5-3B-Instruct2026.01 | 70.4 | |
| GRPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 70.2 | |
| GRPOBackbone=Qwen2.5-3B-Instruct2026.01 | 70.2 | |
| SFT + GRPOBackbone=Qwen2.5-3B-Instruct2026.01 | 70 | |
| Self-ReminderBackbone=R1-Qwen-14B2025.08 | 69.1 | |
| BaseModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 68.8 | |
| Self-RefineBackbone=Qwen2.5-7B-Instruct2026.01 | 68.4 | |
| BaseBackbone=Qwen2.5-3B-Instruct2026.01 | 66.6 | |
| Self-RefineBackbone=Qwen2.5-3B-Instruct2026.01 | 65.6 | |
| SCR-Stage IBackbone=Qwen2.5-3B-Instruct2026.01 | 65.2 | |
| SCR-SFTBackbone=Qwen2.5-3B-Instruct2026.01 | 63.8 | |
| SCR (Ours)Backbone=Llama3.1-8B-Instruct2026.01 | 60.8 | |
| SFT + GRPOBackbone=Llama3.1-8B-Instruct2026.01 | 57.6 | |
| SCR-Stage IBackbone=Llama3.1-8B-Instruct2026.01 | 55.8 | |
| SmoothLLMBackbone=R1-Qwen-32B2025.08 | 53.8 | |
| SCR-SFTBackbone=Llama3.1-8B-Instruct2026.01 | 53.4 | |
| GRPOBackbone=Llama3.1-8B-Instruct2026.01 | 52.2 | |
| Self-ReminderBackbone=R1-Qwen-32B2025.08 | 51.8 | |
| SmoothLLMBackbone=R1-Qwen-14B2025.08 | 49.1 | |
| Qwen2.5-Math-7BModel Type=Base Model2026.02 | 48.1 | |
| BaseModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=22026.01 | 44.5 | |
| BaseBackbone=Llama3.1-8B-Instruct2026.01 | 42.8 | |
| Self-RefineBackbone=Llama3.1-8B-Instruct2026.01 | 13.4 |