Mathematical Reasoning on Countdown 4,5,6-arg held-out difficulties (test)
25.1AccuracySkillFactory -> GRPO
Evaluation results
| Method | Links | |
|---|---|---|
| SkillFactory -> GRPObase_model=Qwen2.5-1.5B-Instruct, training=SFT + GRPO2025.12 | 25.1 | |
| R1 Distill -> GRPObase_model=Qwen2.5-1.5B-Instruct, training=SFT + GRPO2025.12 | 21.2 | |
| RL-Onlybase_model=Qwen2.5-1.5B-Instruct, training=RL (GRPO)2025.12 | 15.8 | |
| BOLT -> GRPObase_model=Qwen2.5-1.5B-Instruct, training=SFT + GRPO2025.12 | 13.7 | |
| R1 Distillbase_model=Qwen2.5-1.5B-Instruct, training=SFT2025.12 | 11.7 | |
| STaR -> GRPObase_model=Qwen2.5-1.5B-Instruct, training=SFT + GRPO2025.12 | 9.7 | |
| SkillFactorybase_model=Qwen2.5-1.5B-Instruct, training=SFT2025.12 | 2.8 | |
| STaRbase_model=Qwen2.5-1.5B-Instruct, training=SFT2025.12 | 2.6 | |
| Qwen2.5 1.5B Instructbase_model=Qwen2.5-1.5B-Instruct, training=None (Baseline)2025.12 | 1.9 | |
| BOLTbase_model=Qwen2.5-1.5B-Instruct, training=SFT2025.12 | 0.5 |