List your company
ResearchFormerly WizWand

Mathematical Reasoning on MATH 500 (Pass@k Metrics)

98.5Pass@1CRAFT
60.5470.39580.2590.105May 14, 2025Jul 12, 2025Sep 10, 2025Nov 8, 2025Jan 7, 2026Mar 7, 2026May 6, 2026
Updated 5mo ago

Evaluation results

MethodLinks
CRAFTBackbone=DeepSeek-R1-Distill-Llama-8B2026.0398.5——
BaseBackbone=Qwen3-4B-thinking2026.0395.2——
CRAFTBackbone=Qwen3-4B-thinking2026.0392.4——
SafeKeyBackbone=DeepSeek-R1-Distill-Llama-8B2026.0392——
SFT→RLBase Model=Qwen2.5-Math-7B, Source=verl2026.0492——
BaseBackbone=DeepSeek-R1-Distill-Llama-8B2026.0391.8——
IPOBackbone=DeepSeek-R1-Distill-Llama-8B2026.0391.6——
MELBackbone=Qwen3-14B-Base2026.0290.890.897.2
IPOBackbone=Qwen3-4B-thinking2026.0390.3——
SafeKeyBackbone=Qwen3-4B-thinking2026.0390.1——
SFTBase Model=Qwen2.5-Math-7B, Source=verl2026.0490——
RealSafeBackbone=DeepSeek-R1-Distill-Llama-8B2026.0389.8——
SRFT [13]Base Model=Qwen2.5-Math-7B, Source=reported2026.0489.8——
ReasoningShieldBackbone=DeepSeek-R1-Distill-Llama-8B2026.0389.6——
STARBackbone=DeepSeek-R1-Distill-Llama-8B2026.0389.4——
HPT [15]Base Model=Qwen2.5-Math-7B, Source=reported2026.0489.2——
Prefix-RFT [14]Base Model=Qwen2.5-Math-7B, Source=reported2026.0488.4——
ReLIFT [12]Base Model=Qwen2.5-Math-7B, Source=reported2026.0487.9——
DeepScaleR-1.5B-PreviewZero-shot=true, Fine-tuning Samples=40,0002025.0587.8——
LUFFY [11]Base Model=Qwen2.5-Math-7B, Source=reported2026.0487.6——
SafeChainBackbone=DeepSeek-R1-Distill-Llama-8B2026.0387——
MELBackbone=Qwen3-8B-Base2026.0286.686.796.2
STARBackbone=Qwen3-4B-thinking2026.0386.3——
DRA-GRPO (Ours)Zero-shot=true, Fine-tuning Samples=7,0002025.0586.2——
ReLIFT [12]Base Model=Qwen2.5-Math-7B, Source=reproduction attempt2026.0486.2——
GRPO†Zero-shot=true, Fine-tuning Samples=7,0002025.0586——
o1-previewZero-shot=true2025.0585.5——
Open-RS2Zero-shot=true, Fine-tuning Samples=7,0002025.0585.4——
DRA-DR. GRPO (Ours)Zero-shot=true, Fine-tuning Samples=7,0002025.0585.2——
LUFFY [11]Base Model=Qwen2.5-Math-7B, Source=reproduction attempt2026.0485.2——
RealSafeBackbone=Qwen3-4B-thinking2026.0385.1——
GRPOBackbone=Qwen3-14B-Base2026.028588.3596.4
SafeChainBackbone=Qwen3-4B-thinking2026.0385——
GRPOBackbone=Qwen3-8B-Base2026.0284.486.2895.4
Still-3-1.5B-PreviewZero-shot=true, Fine-tuning Samples=30,0002025.0584.4——
Open-RS3Zero-shot=true, Fine-tuning Samples=7,0002025.0584.4——
Open-RS1Zero-shot=true, Fine-tuning Samples=18,6152025.0583.8——
DR. GRPO†Zero-shot=true, Fine-tuning Samples=7,0002025.0583.4——
DeepSeek-R1-Distill-Qwen-1.5BZero-shot=true, Fine-tuning Samples=Base Model2025.0582.8——
Qwen2.5-7B-SimpleRLZero-shot=true2025.0582.4——
MELBackbone=Qwen3-4B-Base2026.0282.282.393.8
GRPOBackbone=Qwen3-4B-Base2026.0281.882.293
APMPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.0481.6——
BaselineBackbone=Qwen3-14B-Base2026.0280.874.1593.6
Qwen-2.5-Math-7B-InstructZero-shot=true2025.0579.8——
DAPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.0479.8——
Eurus-2-7B-PRIMEZero-shot=true2025.0579.2——
rStar-Math-7BZero-shot=true2025.0578.4——
APMPOBackbone=Qwen2.5-Math-1.5B-Instruct2026.0478——
DAPOBackbone=Qwen2.5-Math-1.5B-Instruct2026.0477.2——
BaselineBackbone=Qwen3-8B-Base2026.027773.491.4
GMPOBackbone=Qwen2.5-Math-1.5B-Instruct2026.0476.6——
GMPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.0476.6——
GRPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.0475.4——
GRPOBackbone=Qwen2.5-Math-1.5B-Instruct2026.0475.2——
BaselineBackbone=Qwen3-4B-Base2026.0274.265.7489.6
BaseBackbone=Qwen2.5-Math-1.5B-Instruct2026.0474.2——
ReasoningShieldBackbone=Qwen3-4B-thinking2026.0373.9——
INFOTREE2026.0569.8——
APMPOBackbone=Qwen2.5-3B-Instruct2026.0468.4——
DAPOBackbone=Qwen2.5-3B-Instruct2026.0467.6——
Tree-GRPO2026.0567.2——
GMPOBackbone=Qwen2.5-3B-Instruct2026.0466.8——
GRPOBackbone=Qwen2.5-3B-Instruct2026.0466——
Llama-3.1-70B-InstructZero-shot=true2025.0564.6——
BaseBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.0464.6——
Flat GRPO2026.0563.1——
BaseBackbone=Qwen2.5-3B-Instruct2026.0462——