Mathematical Reasoning on BRUMO 2025 (test)
75.7Pass@1 AccuracyRA-RFT
Evaluation results
| Method | Links | |
|---|---|---|
| RA-RFTModel Scale=Qwen3-4B, Temperature=1.0, Maximum generation length=32,768, Sample count (k)=322026.06 | 75.7 | |
| GRPOModel Scale=Qwen3-4B, Temperature=1.0, Maximum generation length=32,768, Sample count (k)=322026.06 | 69.8 | |
| Base (Instruct)Model Scale=Qwen3-4B, Temperature=1.0, Maximum generation length=32,768, Sample count (k)=322026.06 | 65.5 | |
| RA-RFTModel Scale=Qwen3-1.7B, Temperature=1.0, Maximum generation length=32,768, Sample count (k)=322026.06 | 57.4 | |
| PPCVBase Model=DeepSeek-R1-Distill-Llama-70B2026.02 | 56.66 | |
| GRPOModel Scale=Qwen3-1.7B, Temperature=1.0, Maximum generation length=32,768, Sample count (k)=322026.06 | 54.8 | |
| QuestAModel Scale=Qwen3-1.7B, Temperature=1.0, Maximum generation length=32,768, Sample count (k)=322026.06 | 52.6 | |
| Base (Instruct)Model Scale=Qwen3-1.7B, Temperature=1.0, Maximum generation length=32,768, Sample count (k)=322026.06 | 50.9 | |
| Phi-DecodingBase Model=DeepSeek-R1-Distill-Llama-70B2026.02 | 48 | |
| Predictive DecodingBase Model=DeepSeek-R1-Distill-Llama-70B2026.02 | 44.66 | |
| Chain-of-ThoughtBase Model=DeepSeek-R1-Distill-Llama-70B2026.02 | 43.33 |