Mathematical Reasoning on AIME 24 (Acc@8, Pass@8)
93.33Pass Rate@8DeepSeek-R1-Distill-Qwen-32B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeek-R1-Distill-Qwen-32B2026.05 | 93.33 | 80.83 | — | |
| SkyWork-OR1-Math-7B2026.05 | 83.33 | 67.5 | — | |
| GRPOStudent Model=DeepSeek-R1-Distill-Qwen-7B, Teacher Model=DeepSeek-R1-Distill-Qwen-32B2026.05 | 83.33 | 67.5 | — | |
| JSDStudent Model=DeepSeek-R1-Distill-Qwen-7B, Teacher Model=DeepSeek-R1-Distill-Qwen-32B2026.05 | 83.33 | 60.83 | — | |
| REOPOLDStudent Model=DeepSeek-R1-Distill-Qwen-7B, Teacher Model=DeepSeek-R1-Distill-Qwen-32B2026.05 | 83.33 | 57.08 | — | |
| LGRStudent Model=DeepSeek-R1-Distill-Qwen-7B, Teacher Model=DeepSeek-R1-Distill-Qwen-32B2026.05 | 83.33 | 61.67 | — | |
| OPD (9k)Student Model=DeepSeek-R1-Distill-Qwen-1.5B, Teacher Model=SkyWork-OR1-Math-7B2026.05 | 80 | 45.83 | — | |
| LGR (topk)Student Model=DeepSeek-R1-Distill-Qwen-1.5B, Teacher Model=SkyWork-OR1-Math-7B2026.05 | 80 | 44.17 | — | |
| Base ModelStudent Model=DeepSeek-R1-Distill-Qwen-7B2026.05 | 80 | 52.08 | — | |
| OPD (topk)Student Model=DeepSeek-R1-Distill-Qwen-7B, Teacher Model=DeepSeek-R1-Distill-Qwen-32B2026.05 | 80 | 57.5 | — | |
| LGR (topk)Student Model=DeepSeek-R1-Distill-Qwen-7B, Teacher Model=DeepSeek-R1-Distill-Qwen-32B2026.05 | 80 | 61.67 | — | |
| OPD (9k)Student Model=DeepSeek-R1-Distill-Qwen-7B, Teacher Model=DeepSeek-R1-Distill-Qwen-32B2026.05 | 76.67 | 62.08 | — | |
| LGRStudent Model=DeepSeek-R1-Distill-Qwen-1.5B, Teacher Model=SkyWork-OR1-Math-7B2026.05 | 73.33 | 46.67 | — | |
| OPD (topk)Student Model=DeepSeek-R1-Distill-Qwen-1.5B, Teacher Model=SkyWork-OR1-Math-7B2026.05 | 70 | 45.83 | — | |
| GRPOStudent Model=DeepSeek-R1-Distill-Qwen-1.5B, Teacher Model=SkyWork-OR1-Math-7B2026.05 | 66.67 | 38.33 | — | |
| JSDStudent Model=DeepSeek-R1-Distill-Qwen-1.5B, Teacher Model=SkyWork-OR1-Math-7B2026.05 | 66.67 | 38.75 | — | |
| REOPOLDStudent Model=DeepSeek-R1-Distill-Qwen-1.5B, Teacher Model=SkyWork-OR1-Math-7B2026.05 | 60 | 41.67 | — | |
| Base ModelStudent Model=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 50 | 24.17 | — | |
| Dr. GRPOModel=R1-Distill-7B2026.05 | 48.59 | — | — | |
| DAPOModel=R1-Distill-7B2026.05 | 47.83 | — | — | |
| POPOModel=R1-Distill-7B2026.05 | 47.22 | — | — | |
| SAPOModel=R1-Distill-7B2026.05 | 46.71 | — | — | |
| SAPOModel=Qwen-Math-7B2026.05 | 46.67 | — | — | |
| GRPOModel=R1-Distill-7B2026.05 | 46.67 | — | — | |
| Dr. GRPOModel=Qwen-Math-7B2026.05 | 46.45 | — | — | |
| POPOModel=Qwen-Math-7B2026.05 | 45.13 | — | — | |
| DAPOModel=Qwen-Math-7B2026.05 | 44.76 | — | — | |
| GRPOModel=Qwen-Math-7B2026.05 | 43.33 | — | — | |
| SFTModel=R1-Distill-7B2026.05 | 39.15 | — | — | |
| POPOModel=R1-Distill-1.5B2026.05 | 36.67 | — | — | |
| BaseModel=R1-Distill-7B2026.05 | 36.67 | — | — | |
| Dr. GRPOModel=R1-Distill-1.5B2026.05 | 33.33 | — | — | |
| BaseModel=Qwen-Math-7B2026.05 | 33.33 | — | — | |
| SFTModel=Qwen-Math-7B2026.05 | 32.81 | — | — | |
| SAPOModel=R1-Distill-1.5B2026.05 | 30.21 | — | — | |
| GRPOModel=R1-Distill-1.5B2026.05 | 30 | — | — | |
| DAPOModel=R1-Distill-1.5B2026.05 | 29.72 | — | — | |
| POPOModel=Qwen-Math-1.5B2026.05 | 26.67 | — | — | |
| Dr. GRPOModel=Qwen-Math-1.5B2026.05 | 25.18 | — | — | |
| GRPOModel=Qwen-Math-1.5B2026.05 | 23.33 | — | — | |
| SAPOModel=Qwen-Math-1.5B2026.05 | 23.33 | — | — | |
| DAPOModel=Qwen-Math-1.5B2026.05 | 22.85 | — | — | |
| SFTModel=R1-Distill-1.5B2026.05 | 21.07 | — | — | |
| SFTModel=Qwen-Math-1.5B2026.05 | 20.7 | — | — | |
| TTC-NetBackbone=Llama-3-Instruct-7B, Training=SFT2026.03 | 20 | 3.33 | — | |
| BaseModel=Qwen-Math-1.5B2026.05 | 20 | — | — | |
| BaseModel=R1-Distill-1.5B2026.05 | 20 | — | — | |
| + RetNetBackbone=Llama-3-Instruct-7B, Memory Module=RetNet, Training=SFT2026.03 | 13.33 | 2.5 | — | |
| + MesaNetBackbone=Llama-3-Instruct-7B, Memory Module=MesaNet, Training=SFT2026.03 | 10 | 1.25 | — | |
| Full-vocabTeacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 10 | 3.33 | 4,096 | |
| PowerOPDalpha=50, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 10 | 2.92 | 3,283 | |
| PowerOPDalpha=100, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 10 | 2.5 | 3,282 | |
| PowerOPDalpha=500, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 10 | 3.33 | 3,077 | |
| Full FinetuningBackbone=Llama-3-Instruct-7B, Training=SFT2026.03 | 6.67 | 1.67 | — | |
| VanillaTeacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 6.67 | 1.67 | 4,096 | |
| PowerOPDalpha=0.1, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 6.67 | 2.5 | 3,623 | |
| PowerOPDalpha=0.5, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 6.67 | 2.5 | 3,633 | |
| + AttentionBackbone=Llama-3-Instruct-7B, Memory Module=Attention, Training=SFT2026.03 | 3.33 | 0.42 | — | |
| + MambaBackbone=Llama-3-Instruct-7B, Memory Module=Mamba, Training=SFT2026.03 | 3.33 | 0.83 | — | |
| + GDNBackbone=Llama-3-Instruct-7B, Memory Module=GDN, Training=SFT2026.03 | 3.33 | 0.42 | — | |
| VanillaParam=+ Clip, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 3.33 | 0.42 | 3,722 | |
| VanillaParam=+ Tanh, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 3.33 | 0.42 | 3,786 | |
| VanillaParam=+ Z-Score, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 3.33 | 0.42 | 3,691 | |
| PowerOPDalpha=1, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 3.33 | 2.08 | 3,583 | |
| PowerOPDalpha=5, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 3.33 | 2.5 | 3,610 | |
| PowerOPDalpha=10, Teacher=Qwen3-4B, Student=Qwen3-0.6B-Base2026.06 | 3.33 | 2.08 | 3,597 | |
| Base modelBackbone=Llama-3-Instruct-7B2026.03 | 0 | 0 | — | |
| + MLPBackbone=Llama-3-Instruct-7B, Memory Module=MLP, Training=SFT2026.03 | 0 | 0 | — | |
| AKLDistillation track=Qwen3-14B → Qwen3-8B, Family=forward KL family2026.03 | — | 39.8 | — | |
| BaseDistillation track=Qwen3-14B → Qwen3-8B, Family=forward KL family2026.03 | — | 28.7 | — | |
| Forward KL (unweighted)Distillation track=Qwen3-14B → Qwen3-8B, Family=forward KL family2026.03 | — | 35.9 | — | |
| Hard Filter Forward KLDistillation track=Qwen3-14B → Qwen3-8B, Family=forward KL family2026.03 | — | 39.5 | — | |
| PACED Forward KLDistillation track=Qwen3-14B → Qwen3-8B, Family=forward KL family2026.03 | — | 41.6 | — |