Mathematical Reasoning on OlympiadBench (Pass@1, Pass@128, Pass@256)
47.1Pass@1CaSP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CaSPBackbone=Qwen2.5-32B, alpha=0.052025.10 | 47.1 | 74.2 | — | |
| CaSPBackbone=Qwen2.5-32B, alpha=0.012025.10 | 46.2 | 73.6 | — | |
| GRPO w/ Entropy lossBackbone=Qwen2.5-32B2025.10 | 44.8 | 68.4 | — | |
| GRPOBackbone=Qwen2.5-32B2025.10 | 44.5 | 69.3 | — | |
| CaSPBackbone=Qwen2.5-Math-7B2025.10 | 39.8 | — | 77.8 | |
| GRPO w/ Entropy-AdvBackbone=Qwen2.5-Math-7B2025.10 | 39.7 | — | 71.9 | |
| GRPO w/ forking tokensBackbone=Qwen2.5-Math-7B2025.10 | 39.6 | — | 72.4 | |
| KL-CovBackbone=Qwen2.5-Math-7B2025.10 | 39.2 | — | 76.9 | |
| GRPOBackbone=Qwen2.5-Math-7B2025.10 | 39.1 | — | 74.7 | |
| W-REINFORCEBackbone=Qwen2.5-Math-7B2025.10 | 38.1 | — | 77.6 | |
| NSRBackbone=Qwen2.5-Math-7B2025.10 | 37.8 | — | 78.4 | |
| PSRBackbone=Qwen2.5-Math-7B2025.10 | 37.6 | — | 67.7 | |
| GRPO w/ Entropy lossBackbone=Qwen2.5-Math-7B2025.10 | 37 | — | 76.9 | |
| P@k T.Backbone=Qwen2.5-Math-7B2025.10 | 36.6 | — | 78.2 | |
| Base ModelBackbone=Qwen2.5-32B2025.10 | 28.1 | 77.3 | — | |
| Base ModelBackbone=Qwen2.5-Math-7B2025.10 | 25.6 | — | 77 | |
| CaSPBackbone=Llama3.2-3B-Instruct2025.10 | 21 | — | 69.6 | |
| GRPOBackbone=Llama3.2-3B-Instruct2025.10 | 20.1 | — | 67 | |
| NSRBackbone=Llama3.2-3B-Instruct2025.10 | 20 | — | 68 | |
| W-REINFORCEBackbone=Llama3.2-3B-Instruct2025.10 | 19.6 | — | 65.8 | |
| PSRBackbone=Llama3.2-3B-Instruct2025.10 | 18.9 | — | 63.7 | |
| Base ModelBackbone=Llama3.2-3B-Instruct2025.10 | 12.7 | — | 67.1 |