Mathematical Reasoning on MATH (HS, Acc)
82.4AccuracyAthena-PRM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Athena-PRMPolicy Model=Qwen2.5-7B, Selection Strategy=Athena-PRM, N=82025.06 | 82.4 | — | |
| VisualPRM-8BPolicy Model=Qwen2.5-7B, Selection Strategy=VisualPRM-8B, N=82025.06 | 81.6 | — | |
| Athena-ORMPolicy Model=Qwen2.5-7B, Selection Strategy=Athena-ORM, N=82025.06 | 81.1 | — | |
| Self-consistencyPolicy Model=Qwen2.5-7B, Selection Strategy=Self-consistency, N=82025.06 | 79.9 | — | |
| Qwen2.5-7BPolicy Model=Qwen2.5-7B, Selection Strategy=Zero-shot, N=12025.06 | 75.5 | — | |
| Initial large teacherModel=Qwen2.5-14B-Instruct2026.05 | 69 | — | |
| Initial small teacherModel=Qwen2.5-7B-Instruct2026.05 | 64.6 | — | |
| CLPDStudent Model=Qwen-3B, Data Ordering=student-loss2026.05 | 56.6 | — | |
| CLPDStudent Model=Qwen-3B, Data Ordering=expert CoT2026.05 | 56.4 | — | |
| Curriculum learning onlyStudent Model=Qwen-3B, Teacher=large teacher, Data Ordering=expert CoT2026.05 | 55.4 | — | |
| Curriculum learning onlyStudent Model=Qwen-3B, Teacher=large teacher, Data Ordering=student-loss2026.05 | 55.3 | — | |
| Curriculum learning onlyStudent Model=Qwen-3B, Teacher=small teacher, Data Ordering=student-loss2026.05 | 55.2 | — | |
| Standard distillationStudent Model=Qwen-3B, Teacher=small teacher2026.05 | 55 | — | |
| Curriculum learning onlyStudent Model=Qwen-3B, Teacher=small teacher, Data Ordering=expert CoT2026.05 | 55 | — | |
| Progressive distillation onlyStudent Model=Qwen-3B, Teacher=small & large teacher2026.05 | 55 | — | |
| Standard distillationStudent Model=Qwen-3B, Teacher=large teacher2026.05 | 54.6 | — | |
| AegisModel=Qwen2.5, Mixing ratio (r)=0.12026.03 | 47.61 | 12.48 | |
| Initial studentStudent Model=Qwen-3B2026.05 | 47.5 | — | |
| BeavertailsModel=Qwen2.5, Mixing ratio (r)=0.12026.03 | 47.21 | 42.08 | |
| NoneModel=Qwen2.5, Mixing ratio (r)=0.12026.03 | 47.17 | 16.85 | |
| GR-SAPModel=Qwen2.5, Mixing ratio (r)=0.12026.03 | 46.87 | 10.26 | |
| NoneModel=Llama3, Mixing ratio (r)=0.12026.03 | 23.33 | 5.29 | |
| AegisModel=Llama3, Mixing ratio (r)=0.12026.03 | 23.15 | 2.59 | |
| GR-SAPModel=Llama3, Mixing ratio (r)=0.12026.03 | 23.07 | 0.34 | |
| BeavertailsModel=Llama3, Mixing ratio (r)=0.12026.03 | 22.81 | 30.65 | |
| GR-SAPModel=OLMo2, Mixing ratio (r)=0.12026.03 | 17.8 | 0.5 | |
| OriginalModel=OLMo2, Mixing ratio (r)=0.12026.03 | 17.55 | 0.7 | |
| NoneModel=OLMo2, Mixing ratio (r)=0.12026.03 | 17.54 | 1.29 | |
| BeavertailsModel=OLMo2, Mixing ratio (r)=0.12026.03 | 17.49 | 9.48 | |
| AegisModel=OLMo2, Mixing ratio (r)=0.12026.03 | 17.39 | 2.97 | |
| CLPDStudent Model=Llama-3B, Data Ordering=student-loss2026.05 | 15.3 | — | |
| CLPDStudent Model=Llama-3B, Data Ordering=expert CoT2026.05 | 15 | — | |
| Progressive distillation onlyStudent Model=Llama-3B, Teacher=small & large teacher2026.05 | 13.9 | — | |
| NoneModel=Mistral, Mixing ratio (r)=0.12026.03 | 13.63 | 43.83 | |
| GR-SAPModel=Mistral, Mixing ratio (r)=0.12026.03 | 13.57 | 14.01 | |
| Curriculum learning onlyStudent Model=Llama-3B, Teacher=small teacher, Data Ordering=student-loss2026.05 | 13.4 | — | |
| Curriculum learning onlyStudent Model=Llama-3B, Teacher=large teacher, Data Ordering=student-loss2026.05 | 13.4 | — | |
| AegisModel=Mistral, Mixing ratio (r)=0.12026.03 | 13.37 | 13.07 | |
| BeavertailsModel=Mistral, Mixing ratio (r)=0.12026.03 | 13.2 | 45.35 | |
| Standard distillationStudent Model=Llama-3B, Teacher=small teacher2026.05 | 13 | — | |
| Curriculum learning onlyStudent Model=Llama-3B, Teacher=small teacher, Data Ordering=expert CoT2026.05 | 13 | — | |
| Curriculum learning onlyStudent Model=Llama-3B, Teacher=large teacher, Data Ordering=expert CoT2026.05 | 12.9 | — | |
| Standard distillationStudent Model=Llama-3B, Teacher=large teacher2026.05 | 12.8 | — | |
| Initial studentStudent Model=Llama-3B2026.05 | 3.8 | — |