Mathematical Reasoning on MATH (test) (Accuracy, Efficiency, Speedup)
46.38AccuracyGRPO+FIRSTN
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GRPO+FIRSTNGroup Size G=16, Base Model=Llama3.2-3B-Instruct2026.05 | 46.38 | 362.06 | 3.4 | 11,775 | 1.07 | |
| CPPOGroup Size G=16, Base Model=Llama3.2-3B-Instruct2026.05 | 46.19 | 414.48 | -10.6 | 2,408 | 5.25 | |
| PairGroup Size G=16, Base Model=Llama3.2-3B-Instruct2026.05 | 46.14 | 192.74 | 48.6 | 2,249 | 5.62 | |
| GRPOGroup Size G=16, Base Model=Llama3.2-3B-Instruct2026.05 | 46.01 | 374.83 | — | 12,638 | 1 | |
| CPPOGroup Size G=8, Base Model=Llama3.2-3B-Instruct2026.05 | 45.88 | 417.7 | -15.9 | 3,502 | 2.59 | |
| PairGroup Size G=8, Base Model=Llama3.2-3B-Instruct2026.05 | 45.68 | 233 | 35.4 | 3,207 | 2.83 | |
| BPPOGroup Size G=16, Base Model=Llama3.2-3B-Instruct2026.05 | 45.61 | 185.2 | 50.6 | 2,079 | 6.08 | |
| GRPOGroup Size G=8, Base Model=Llama3.2-3B-Instruct2026.05 | 45.33 | 360.54 | — | 9,070 | 1 | |
| BPPOGroup Size G=8, Base Model=Llama3.2-3B-Instruct2026.05 | 45.23 | 190 | 47.3 | 2,788 | 3.25 | |
| GRPO+FIRSTNGroup Size G=8, Base Model=Llama3.2-3B-Instruct2026.05 | 44.91 | 319.77 | 11.3 | 7,904 | 1.15 | |
| Llama3.2-3B-InstructGroup Size G=-2026.05 | 33.12 | — | — | — | — |