Mathematical Reasoning Process Evaluation on PROCESSBENCH
82.9GSM8K AccuracyLlama-3.3-70B-Instruct
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Llama-3.3-70B-InstructSetup=Language Models as Critic Models2025.12 | 82.9 | 59.4 | 46.7 | 43 | 58 | |
| GPT-4o-0806Setup=Language Models as Critic Models2025.12 | 79.2 | 63.6 | 51.4 | 53.5 | 61.9 | |
| Qwen2.5-72B-InstructSetup=Language Models as Critic Models2025.12 | 76.2 | 61.8 | 54.6 | 52.2 | 61.2 | |
| Llama-3.1-70B-InstructSetup=Language Models as Critic Models2025.12 | 74.9 | 48.2 | 46.7 | 41 | 52.7 | |
| Step ConsistencySetup=PRM (ours)2025.12 | 72 | 67.9 | 64.5 | 65.7 | 67.5 | |
| Skywork-PRM-7BSetup=Open-source Process Reward Models2025.12 | 70.8 | 53.6 | 22.9 | 21 | 42.1 | |
| Meta Critic + Outcome ConsistencySetup=PRM (ours)2025.12 | 70.2 | 68.7 | 63.4 | 65.2 | 66.9 | |
| Reference GuidedSetup=PRM (ours)2025.12 | 70.2 | 67.4 | 63.7 | 64.1 | 66.4 | |
| Meta Critic + Outcome ConsistencySetup=PRM-CoT (ours)2025.12 | 70 | 67.1 | 59.2 | 58.3 | 63.7 | |
| Outcome ConsistencySetup=PRM-CoT (ours)2025.12 | 69.5 | 68.7 | 61.4 | 60.5 | 65 | |
| Qwen2.5-14B-InstructSetup=Language Models as Critic Models2025.12 | 69.3 | 53.3 | 45 | 41.3 | 52.2 | |
| Outcome ConsistencySetup=PRM (ours)2025.12 | 68.3 | 68 | 66.6 | 63.3 | 66.6 | |
| Qwen2.5-Math-7B-PRM800KSetup=Open-source Process Reward Models2025.12 | 68.2 | 62.6 | 50.7 | 44.3 | 56.5 | |
| Meta CriticSetup=PRM (ours)2025.12 | 68 | 67.2 | 62.4 | 65.5 | 65.8 | |
| Step ConsistencySetup=PRM-CoT (ours)2025.12 | 67.6 | 70.3 | 63.9 | 61 | 65.7 | |
| Single VerificationSetup=PRM (ours)2025.12 | 67 | 65.5 | 61.2 | 62 | 63.9 | |
| Reference GuidedSetup=PRM-CoT (ours)2025.12 | 66.1 | 66.6 | 60.2 | 59.8 | 63.2 | |
| Qwen2.5-Math-72B-InstructSetup=Language Models as Critic Models2025.12 | 65.8 | 52.1 | 32.5 | 31.7 | 45.5 | |
| Qwen2.5-32B-InstructSetup=Language Models as Critic Models2025.12 | 65.6 | 53.1 | 40 | 38.3 | 49.3 | |
| Meta CriticSetup=PRM-CoT (ours)2025.12 | 65.3 | 66 | 60.4 | 56.6 | 62.1 | |
| Single VerificationSetup=PRM-CoT (ours)2025.12 | 62.5 | 63 | 56.8 | 57 | 59.8 | |
| Skywork-PRM-1.5BSetup=Open-source Process Reward Models2025.12 | 59 | 48 | 19.3 | 19.2 | 36.4 | |
| RLHFlow-PRM-Mistral-8BSetup=Open-source Process Reward Models2025.12 | 50.4 | 33.4 | 13.8 | 15.8 | 28.4 | |
| Math-Shepherd-PRM-7BSetup=Open-source Process Reward Models2025.12 | 47.9 | 29.5 | 24.8 | 23.8 | 31.5 | |
| RLHFlow-PRM-Deepseek-8BSetup=Open-source Process Reward Models2025.12 | 38.8 | 33.8 | 16.9 | 16.9 | 26.6 | |
| Qwen2.5-7B-InstructSetup=Language Models as Critic Models2025.12 | 36.5 | 36.6 | 29.7 | 27.4 | 32.6 | |
| Qwen2.5-Math-7B-InstructSetup=Language Models as Critic Models2025.12 | 26.8 | 25.7 | 14.2 | 12.7 | 19.9 | |
| Llama-3.1-8B-InstructSetup=Language Models as Critic Models2025.12 | 10.9 | 5.1 | 2.8 | 1.6 | 5.1 |