Mathematical Reasoning Suite (AMC, AIME 2024, AIME 2025, Minerva, MATH, Olympiad) (various test/val)
62.1Average ScoreCPMöbius
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CPMöbiusBackbone=OpenMath-Nemotron-1.5B2026.02 | 62.1 | 57 | 87.5 | 54.9 | 46.9 | 24.3 | 91.2 | 67.9 | — | — | — | — | — | — | — | |
| RENTBackbone=OpenMath-Nemotron-1.5B2026.02 | 61.7 | 56.5 | 87.7 | 55 | 46 | 24.2 | 90.7 | 66.7 | — | — | — | — | — | — | — | |
| Base ModelBackbone=OpenMath-Nemotron-1.5B2026.02 | 59.5 | 54.9 | 82.3 | 55.6 | 43.3 | 25.1 | 89.4 | 61 | — | — | — | — | — | — | — | |
| VI-CuRLBackbone=Qwen2.5-Math-7B, Reward Strategy=Verifier-Free: Majority Vote2026.02 | 43.4 | — | — | 33.9 | 1,440 | — | — | — | — | — | — | 65.4 | 79.8 | 25.9 | 41.2 | |
| VI-CuRLBackbone=Qwen2.5-Math-7B, Reward Strategy=Oracle Reward (w. Verifier)2026.02 | 42.9 | — | — | 29.9 | 1,390 | — | — | — | — | — | — | 63.8 | 78.8 | 26.3 | 45 | |
| No CurriculumBackbone=Qwen2.5-Math-7B, Reward Strategy=Verifier-Free: Majority Vote2026.02 | 41.8 | — | — | 29 | 1,400 | — | — | — | — | — | — | 64.8 | 80.5 | 22.8 | 39.5 | |
| No CurriculumBackbone=Qwen2.5-Math-7B, Reward Strategy=Oracle Reward (w. Verifier)2026.02 | 41 | — | — | 29.2 | 1,350 | — | — | — | — | — | — | 62.8 | 78.9 | 24 | 37.8 | |
| CPMöbiusBackbone=Qwen2.5-Math-7B-Instruct2026.02 | 40.7 | 38.4 | 55.6 | 11.8 | 9.6 | 44.9 | 84.2 | 38.3 | — | — | — | — | — | — | — | |
| AdaRFTBackbone=Qwen2.5-Math-7B, Reward Strategy=Oracle Reward (w. Verifier)2026.02 | 40.1 | — | — | 26.9 | 1,290 | — | — | — | — | — | — | 63 | 76.6 | 22.5 | 39.1 | |
| RENTBackbone=Qwen2.5-Math-7B-Instruct2026.02 | 39.2 | 37.6 | 53.1 | 10.8 | 9.9 | 38.8 | 83.8 | 38.8 | — | — | — | — | — | — | — | |
| VI-CuRLBackbone=Qwen2.5-Math-7B, Reward Strategy=Verifier-Free: Entropy2026.02 | 39 | — | — | 25 | 1,310 | — | — | — | — | — | — | 63 | 76.8 | 19.1 | 37 | |
| VCRLBackbone=Qwen2.5-Math-7B, Reward Strategy=Oracle Reward (w. Verifier)2026.02 | 37.1 | — | — | 22 | 1,110 | — | — | — | — | — | — | 57 | 75.1 | 22.1 | 35.6 | |
| R-Zero (Iter 3)Backbone=Qwen2.5-Math-7B-Instruct2026.02 | 36.9 | 34.2 | 50.5 | 9.5 | 7.4 | 32.7 | 83.3 | 38.1 | — | — | — | — | — | — | — | |
| Base ModelBackbone=Qwen2.5-Math-7B-Instruct2026.02 | 35.8 | 33 | 49.2 | 9 | 6.3 | 34.6 | 78 | 37.4 | — | — | — | — | — | — | — | |
| No CurriculumBackbone=Qwen2.5-Math-7B, Reward Strategy=Verifier-Free: Entropy2026.02 | 34.8 | — | — | 13.7 | 720 | — | — | — | — | — | — | 55.7 | 74.4 | 24.8 | 32.9 | |
| CPMöbiusBackbone=Qwen2.5-Math-1.5B2026.02 | 28.8 | 26.8 | 39.4 | 9.8 | 5.4 | 28 | 63.1 | 26.9 | — | — | — | — | — | — | — | |
| R-Zero (Iter 3)Backbone=Qwen2.5-Math-1.5B2026.02 | 27.1 | 24.7 | 39.2 | 9.8 | 5 | 19.3 | 62.4 | 26.8 | — | — | — | — | — | — | — | |
| RENTBackbone=Qwen2.5-Math-1.5B2026.02 | 27.1 | 24.7 | 39.3 | 10 | 5 | 19 | 62.2 | 27.1 | — | — | — | — | — | — | — | |
| A2DBackbone=Qwen2.5-7B-Instruct, Training Approach=A2D2026.01 | 26.5 | — | 53.4 | 20 | 17.1 | 28.9 | 75.6 | — | 6.6 | 8.4 | 1.8 | — | — | — | — | |
| A2DBackbone=Qwen2.5-Math-7B-Instruct, Training Approach=A2D2026.01 | 26 | — | 58.8 | 11.7 | 11.3 | 25.6 | 80.3 | — | 9 | 8.4 | 3 | — | — | — | — | |
| Base (No RL)Backbone=Qwen2.5-Math-7B2026.02 | 25.2 | — | — | 12.7 | 580 | — | — | — | — | — | — | 44.4 | 52 | 9.8 | 26.4 | |
| GRPOBackbone=Qwen2.5-Math-7B-Instruct, Training Approach=GRPO2026.01 | 25.1 | — | 58.1 | 10.4 | 8.8 | 24.6 | 79.7 | — | 8.3 | 8.7 | 2 | — | — | — | — | |
| Scaf-GRPOBackbone=Qwen2.5-7B-Instruct, Training Approach=Scaf-GRPO2026.01 | 24.1 | — | 54.7 | 14.6 | 9.6 | 25.1 | 74.5 | — | 5.5 | 5.6 | 2.8 | — | — | — | — | |
| BaselineBackbone=Qwen2.5-Math-7B-Instruct, Training Approach=Baseline2026.01 | 23.8 | — | 56.3 | 10.8 | 6.7 | 23.4 | 78.5 | — | 6.4 | 5.9 | 2 | — | — | — | — | |
| CPMöbiusBackbone=OctoThinker-3B-Hybrid-Zero2026.02 | 23.6 | 22 | 28 | 4.8 | 1.7 | 22.1 | 60.4 | 24.7 | — | — | — | — | — | — | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Training Approach=GRPO2026.01 | 23.4 | — | 52.8 | 15 | 8.3 | 24.8 | 73.3 | — | 5.3 | 4.6 | 2.9 | — | — | — | — | |
| Base ModelBackbone=Qwen2.5-Math-1.5B2026.02 | 23.3 | 19.8 | 34.6 | 6.2 | 2.8 | 16.3 | 56.2 | 23.4 | — | — | — | — | — | — | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Training Approach=GRPO, n_rollout=642026.01 | 23.2 | — | 51.9 | 15 | 6.7 | 25.3 | 73.9 | — | 5.4 | 4.6 | 3 | — | — | — | — | |
| RENTBackbone=OctoThinker-3B-Hybrid-Zero2026.02 | 23 | 21.7 | 29.2 | 7.3 | 2.1 | 15 | 60.2 | 24.1 | — | — | — | — | — | — | — | |
| Base ModelBackbone=OctoThinker-3B-Hybrid-Zero2026.02 | 21.3 | 20.6 | 24.6 | 3.9 | 1.7 | 16.3 | 57.9 | 23.4 | — | — | — | — | — | — | — | |
| LUFFYBackbone=Qwen2.5-7B-Instruct, Training Approach=LUFFY2026.01 | 21.1 | — | 47.2 | 10.4 | 8.8 | 22.4 | 69.2 | — | 3.9 | 4.4 | 2.8 | — | — | — | — | |
| BaselineBackbone=Qwen2.5-7B-Instruct, Training Approach=Baseline2026.01 | 20.9 | — | 47.5 | 9.6 | 6.3 | 22.9 | 70.8 | — | 3.4 | 4.8 | 2.1 | — | — | — | — | |
| R-Zero (Iter 3)Backbone=OctoThinker-3B-Hybrid-Zero2026.02 | 20.5 | 19.5 | 25.9 | 2 | 0.3 | 14.6 | 58.1 | 22.3 | — | — | — | — | — | — | — | |
| SFT w/ CoTBackbone=Qwen2.5-7B-Instruct, Training Approach=SFT w/ CoT2026.01 | 18.3 | — | 41.3 | 7.1 | 2.9 | 24.4 | 63.6 | — | 2.8 | 2.9 | 1.4 | — | — | — | — | |
| SFT w/ CoT & SQBackbone=Qwen2.5-7B-Instruct, Training Approach=SFT w/ CoT & SQ2026.01 | 17.7 | — | 40 | 5.8 | 1.3 | 23.3 | 62.7 | — | 2.5 | 4.3 | 1.4 | — | — | — | — | |
| VI-CuRLBackbone=Llama 3.2 3B-Instruct, Reward Strategy=Verifier-Free: Majority Vote2026.02 | 16.3 | — | — | 5.2 | 80 | — | — | — | — | — | — | 19.6 | 46.2 | 10.8 | 15.1 | |
| VI-CuRLBackbone=Llama 3.2 3B-Instruct, Reward Strategy=Verifier-Free: Entropy2026.02 | 15.9 | — | — | 6.2 | 80 | — | — | — | — | — | — | 21.6 | 42.6 | 9.7 | 14.7 | |
| No CurriculumBackbone=Llama 3.2 3B-Instruct, Reward Strategy=Oracle Reward (w. Verifier)2026.02 | 15.7 | — | — | 6 | 70 | — | — | — | — | — | — | 24.3 | 40.4 | 8.2 | 14.3 | |
| VI-CuRLBackbone=Llama 3.2 3B-Instruct, Reward Strategy=Oracle Reward (w. Verifier)2026.02 | 15.3 | — | — | 5.1 | 80 | — | — | — | — | — | — | 19.8 | 43 | 9.4 | 13.6 | |
| A2DBackbone=LLaMA3.2-3B-Instruct, Training Approach=A2D2026.01 | 15.2 | — | 30.9 | 13.8 | 2.1 | 15.3 | 52.6 | — | 2 | 3.1 | 1.6 | — | — | — | — | |
| No CurriculumBackbone=Llama 3.2 3B-Instruct, Reward Strategy=Verifier-Free: Majority Vote2026.02 | 15.2 | — | — | 5.7 | 60 | — | — | — | — | — | — | 18.5 | 41.9 | 8.8 | 15.4 | |
| GRPOBackbone=LLaMA3.2-3B-Instruct, Training Approach=GRPO, n_rollout=642026.01 | 14.4 | — | 30 | 10.8 | 1.7 | 14.6 | 52.5 | — | 1.1 | 2.8 | 1.3 | — | — | — | — | |
| A2DBackbone=LLaMA3.1-8B-Instruct, Training Approach=A2D2026.01 | 14.4 | — | 30.6 | 8.3 | 1.3 | 19.9 | 50.6 | — | 1.4 | 1.6 | 1.6 | — | — | — | — | |
| AdaRFTBackbone=Llama 3.2 3B-Instruct, Reward Strategy=Oracle Reward (w. Verifier)2026.02 | 14.4 | — | — | 4.5 | 70 | — | — | — | — | — | — | 15.7 | 44 | 8.5 | 12.8 | |
| Scaf-GRPOBackbone=LLaMA3.2-3B-Instruct, Training Approach=Scaf-GRPO2026.01 | 14.2 | — | 30.6 | 11.3 | 0.8 | 15.3 | 51.2 | — | 0.8 | 2.6 | 0.9 | — | — | — | — | |
| GRPOBackbone=LLaMA3.2-3B-Instruct, Training Approach=GRPO2026.01 | 14 | — | 29.4 | 11.3 | 0.8 | 13.4 | 52.5 | — | 0.6 | 2.5 | 1.3 | — | — | — | — | |
| GRPOBackbone=LLaMA3.1-8B-Instruct, Training Approach=GRPO2026.01 | 13.7 | — | 28.8 | 7.1 | 0 | 19.6 | 50.3 | — | 0.8 | 1.6 | 1 | — | — | — | — | |
| VCRLBackbone=Llama 3.2 3B-Instruct, Reward Strategy=Oracle Reward (w. Verifier)2026.02 | 13.1 | — | — | 3 | 40 | — | — | — | — | — | — | 17 | 37 | 7.8 | 13.2 | |
| Base (No RL)Backbone=Llama 3.2 3B-Instruct2026.02 | 12.6 | — | — | 5.7 | 60 | — | — | — | — | — | — | 17.2 | 34.8 | 4.8 | 12.7 | |
| SFT w/ CoTBackbone=LLaMA3.2-3B-Instruct, Training Approach=SFT w/ CoT2026.01 | 10.1 | — | 22.5 | 2.1 | 1.7 | 9.6 | 41.5 | — | 0.5 | 2.1 | 0.6 | — | — | — | — | |
| SFT w/ CoT & SQBackbone=LLaMA3.2-3B-Instruct, Training Approach=SFT w/ CoT & SQ2026.01 | 10.1 | — | 21.9 | 3.8 | 0.4 | 9.9 | 41.6 | — | 0.8 | 1.5 | 1 | — | — | — | — | |
| BaselineBackbone=LLaMA3.1-8B-Instruct, Training Approach=Baseline2026.01 | 7.7 | — | 13.8 | 2.5 | 0 | 10.9 | 33.3 | — | 0.1 | 0.4 | 0.6 | — | — | — | — | |
| LUFFYBackbone=LLaMA3.2-3B-Instruct, Training Approach=LUFFY2026.01 | 6 | — | 9.7 | 1.7 | 0 | 6.6 | 26.4 | — | 0.6 | 1.6 | 1.1 | — | — | — | — | |
| BaselineBackbone=LLaMA3.2-3B-Instruct, Training Approach=Baseline2026.01 | 5.9 | — | 10.9 | 1.7 | 0 | 6.6 | 26.8 | — | 0.1 | 1 | 0.4 | — | — | — | — | |
| No CurriculumBackbone=Llama 3.2 3B-Instruct, Reward Strategy=Verifier-Free: Entropy2026.02 | 1.7 | — | — | 0.6 | 0 | — | — | — | — | — | — | 3 | 3.2 | 2.1 | 1.2 |