Mathematical reasoning on OlympiadBench (Pass@1 accuracy %)
47.4Pass@1 AccuracyISPO
Evaluation results
| Method | Links | |
|---|---|---|
| ISPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 47.4 | |
| PACRBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 46.1 | |
| Dr. GRPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 45.2 | |
| ISPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 43.7 | |
| Scaf-GRPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 43.3 | |
| ProGRPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 42.7 | |
| PREPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 41.6 | |
| Vanilla GRPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 41.3 | |
| Scaf-GRPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 40.8 | |
| ISPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 40.3 | |
| PREPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 39.9 | |
| Dr. GRPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 39 | |
| PACRBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 39 | |
| Scaf-GRPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 36.6 | |
| Vanilla GRPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 35.6 | |
| Vanilla GRPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 34.8 | |
| PREPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 32 | |
| Base modelBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 24.6 | |
| Base modelBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 22.8 | |
| Base modelBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 16.6 |