Mathematical Reasoning on OlyBench (Pass@1 accuracy)
47.1Pass@1 AccuracyrStar
Evaluation results
| Method | Links | |
|---|---|---|
| rStarFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Non-comparable Baselines2025.08 | 47.1 | |
| LUFFYFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Non-comparable Baselines2025.08 | 45.6 | |
| PrismBase model=Qwen3-4B-Base2026.03 | 45.58 | |
| R-ZeroBase model=Qwen3-4B-Base2026.03 | 44.59 | |
| ReLIFTFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Non-comparable Baselines2025.08 | 44.3 | |
| R-FewBase model=Qwen3-4B-Base, Selection percentage=5%2026.03 | 42.8 | |
| SPICEBase model=Qwen3-4B-Base2026.03 | 42.7 | |
| R-FewBase model=Qwen3-4B-Base, Selection percentage=1%2026.03 | 42.4 | |
| Simple-RLFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Comparable Baselines (zero setting or less expert data)2025.08 | 40.7 | |
| InstructFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Non-comparable Baselines2025.08 | 39.9 | |
| DPO-R1Fine-tuning base=Qwen2.5-Math-7B, Comparison Group=Comparable Baselines (zero setting or less expert data)2025.08 | 38.8 | |
| Absolute ZeroBase model=Qwen3-4B-Base2026.03 | 38.5 | |
| DPO-VPFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Comparable Baselines (zero setting or less expert data)2025.08 | 38.2 | |
| DPO-LoRRFine-tuning base=Qwen2.5-Math-7B, Iteration=iter32025.08 | 37.2 | |
| DPO-LoRRFine-tuning base=Qwen2.5-Math-7B, Iteration=iter22025.08 | 37 | |
| PRIMEFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Non-comparable Baselines2025.08 | 35.6 | |
| S1Fine-tuning base=Qwen2.5-Math-7B, Comparison Group=Comparable Baselines (zero setting or less expert data)2025.08 | 35.1 | |
| DPO-LoRRFine-tuning base=Qwen2.5-Math-7B, Iteration=iter12025.08 | 35.1 | |
| Qwen3-4B-Base2026.03 | 34.8 | |
| LIMOFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Comparable Baselines (zero setting or less expert data)2025.08 | 32.3 | |
| PURE-VRFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Comparable Baselines (zero setting or less expert data)2025.08 | 28.1 | |
| Qwen2.5Fine-tuning base=Qwen2.5-Math-7B, Comparison Group=Base2025.08 | 24.1 |