Mathematical Reasoning on Olympiad Bench (average@16 accuracy)
47.8Average@16 AccuracyPROF-BOTH
Evaluation Results
| Method | Links | |
|---|---|---|
| PROF-BOTHModel=Qwen2.5-Math-7B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 47.8 | |
| GRPOModel=Qwen2.5-Math-7B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 45.5 | |
| BlendModel=Qwen2.5-Math-7B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 45 | |
| PROF-POSModel=Qwen2.5-Math-7B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 45 | |
| PROF-POSModel=Qwen2.5-Math-1.5B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 36.1 | |
| PROF-BOTHModel=Qwen2.5-Math-1.5B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 36.1 | |
| GRPOModel=Qwen2.5-Math-1.5B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 33 | |
| BlendModel=Qwen2.5-Math-1.5B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 31.1 | |
| PROF-POSModel=LLaMA-3.2-3B-instruct, Temperature=1.0, Evaluation Protocol=average@162025.09 | 19.8 | |
| BaseModel=Qwen2.5-Math-7B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 19.2 | |
| BaseModel=Qwen2.5-Math-1.5B-base, Temperature=1.0, Evaluation Protocol=average@162025.09 | 19.1 | |
| GRPOModel=LLaMA-3.2-3B-instruct, Temperature=1.0, Evaluation Protocol=average@162025.09 | 17.9 | |
| PROF-BOTHModel=LLaMA-3.2-3B-instruct, Temperature=1.0, Evaluation Protocol=average@162025.09 | 17.3 | |
| BlendModel=LLaMA-3.2-3B-instruct, Temperature=1.0, Evaluation Protocol=average@162025.09 | 9.9 | |
| BaseModel=LLaMA-3.2-3B-instruct, Temperature=1.0, Evaluation Protocol=average@162025.09 | 6.1 |