Process-level Error Localization on PROCESSBENCH
88GSM8K AccuracyQwQ-32B-Preview
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| QwQ-32B-PreviewEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Reasoning Models (Self-Consistency)2025.03 | 88 | 78.7 | 57.8 | 61.3 | — | 71.5 | 20.6 | |
| Qwen2.5-Math-PRM-72BEvaluation Protocol=Direct Process Evaluator (PRMs)2025.03 | 87.3 | 80.6 | 74.3 | 71.1 | — | 78.3 | — | |
| QwQ-32B-PreviewEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation + Self-Consistency2025.03 | 86.8 | 85.7 | 79 | 78 | — | 82.4 | 31.5 | |
| DeepSeek-R1-Distill-Qwen-32BEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation + Self-Consistency2025.03 | 86.6 | 85.4 | 78.9 | 80.3 | — | 82.8 | 7.3 | |
| QwQ-32BEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation + Self-Consistency2025.03 | 85 | 85.6 | 79.4 | 78 | — | 82 | 7.5 | |
| DeepSeek-R1-Distill-Qwen-32BEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Reasoning Models2025.03 | 83.9 | 78.1 | 72.4 | 67.7 | — | 75.5 | 0 | |
| Qwen2.5-Math-PRM-7BEvaluation Protocol=Direct Process Evaluator (PRMs)2025.03 | 82.4 | 77.6 | 67.5 | 66.3 | — | 73.5 | — | |
| DeepSeek-R1-Distill-Qwen-32BEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Reasoning Models (Self-Consistency)2025.03 | 82.2 | 80.4 | 76.2 | 72.5 | — | 77.8 | 2.3 | |
| QwQ-32B-PreviewEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation2025.03 | 81.7 | 79.3 | 70.3 | 69.8 | — | 75.3 | 24.4 | |
| QwQ-32BEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation2025.03 | 81.5 | 83.6 | 76.8 | 75.1 | — | 79.3 | 4.8 | |
| QwQ-32BEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Reasoning Models (Self-Consistency)2025.03 | 81 | 78.8 | 74.4 | 72.8 | — | 76.8 | 2.3 | |
| DeepSeek-R1-Distill-Qwen-32BEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation2025.03 | 80.3 | 82.2 | 77 | 75 | — | 78.6 | 3.1 | |
| DeepSeek-R1-Distill-Qwen-7BEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation + Self-Consistency2025.03 | 80.1 | 75.1 | 69.3 | 70.4 | — | 73.7 | 19.2 | |
| QwQ-32BEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Reasoning Models2025.03 | 79.5 | 77.5 | 71.5 | 69.4 | — | 74.5 | 0 | |
| QwQ-32B-PreviewEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Reasoning Models2025.03 | 77.5 | 58.9 | 31.2 | 35.8 | — | 50.9 | 0 | |
| Qwen2.5-72B-InstructEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Instruction-tuned Models2025.03 | 76.2 | 61.8 | 54.6 | 52.2 | — | 61.2 | — | |
| DeepSeek-R1-Distill-Qwen-7BEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation2025.03 | 75.5 | 67.3 | 59.8 | 56.6 | — | 64.8 | 10.3 | |
| Qwen2.5-32B-InstructEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation2025.03 | 70.1 | 61.7 | 54.2 | 53.9 | — | 60 | 15 | |
| DeepSeek-R1-Distill-Qwen-7BEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Reasoning Models (Self-Consistency)2025.03 | 69.3 | 67.9 | 54.8 | 51.5 | — | 60.9 | 6.4 | |
| Qwen2.5-Math-7B-PRM800KEvaluation Protocol=Direct Process Evaluator (PRMs)2025.03 | 68.4 | 62.5 | 50.4 | 43.6 | — | 56.2 | — | |
| DeepSeek-R1-Distill-Qwen-7BEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Reasoning Models2025.03 | 68.3 | 61.1 | 48.3 | 40.4 | — | 54.5 | 0 | |
| Qwen2.5-32B-InstructEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Instruction-tuned Models2025.03 | 63.8 | 47.5 | 35.9 | 32.7 | — | 45 | 0 | |
| Q-RMBase Model=Qwen3-8B base2026.04 | 61 | 45.2 | 39.5 | 35.3 | 45.3 | — | — | |
| IPVRMBase Model=Qwen3-8B base2026.04 | 57.2 | 47.8 | 38.8 | 38.6 | 45.6 | — | — | |
| IPVRMBase Model=Qwen3-0.6B base2026.04 | 52.8 | 42.1 | 30 | 31.3 | 39.1 | — | — | |
| DPO-RMBase Model=Qwen3-8B base2026.04 | 52.2 | 48.2 | 36 | 33.3 | 42.4 | — | — | |
| Implicit PRMBase Model=Qwen3-8B base2026.04 | 51.6 | 47.8 | 36.4 | 33.1 | 42.2 | — | — | |
| IPVRMBase Model=Llama3.2-3B2026.04 | 48.1 | 38.7 | 32.1 | 34.2 | 38.3 | — | — | |
| Q-RMBase Model=Qwen3-0.6B base2026.04 | 47.2 | 38.9 | 31.7 | 35 | 38.2 | — | — | |
| IPVRMBase Model=Llama3.2-1B2026.04 | 42.7 | 35.2 | 28.7 | 29.2 | 34 | — | — | |
| Q-RMBase Model=Llama3.2-3B2026.04 | 42.7 | 37.5 | 34.4 | 36.1 | 37.7 | — | — | |
| DPO-RMBase Model=Llama3.2-3B2026.04 | 41.2 | 33.9 | 25.1 | 27.4 | 31.9 | — | — | |
| Implicit PRMBase Model=Llama3.2-3B2026.04 | 39.9 | 33.9 | 23.4 | 22.9 | 30 | — | — | |
| DPO-RMBase Model=Llama3.2-1B2026.04 | 38.7 | 27.9 | 21.2 | 22 | 27.5 | — | — | |
| Implicit PRMBase Model=Llama3.2-1B2026.04 | 37.8 | 28.9 | 21.4 | 23.5 | 27.9 | — | — | |
| Llama-3.1-8B-InstructEvaluation Protocol=Reasoning Process Evaluator (Ours), Scaling Strategy=Multi-step Process Evaluation2025.03 | 35.2 | 22.8 | 12.6 | 17.9 | — | 22.1 | 7.3 | |
| EndoRMBase Model=Qwen3-8B base2026.04 | 35 | 30.9 | 28.1 | 29.3 | 30.8 | — | — | |
| EndoRMBase Model=Qwen3-0.6B base2026.04 | 33.7 | 30.1 | 26.9 | 31.5 | 30.6 | — | — | |
| EndoRMBase Model=Llama3.2-1B2026.04 | 33.5 | 29.2 | 26 | 32 | 30.2 | — | — | |
| EndoRMBase Model=Llama3.2-3B2026.04 | 32.5 | 28.9 | 23.9 | 29.7 | 28.8 | — | — | |
| DPO-RMBase Model=Qwen3-0.6B base2026.04 | 32.1 | 32.7 | 23.5 | 22.9 | 27.8 | — | — | |
| Implicit PRMBase Model=Qwen3-0.6B base2026.04 | 30.3 | 31 | 21.5 | 18.8 | 25.4 | — | — | |
| Q-RMBase Model=Llama3.2-1B2026.04 | 29.7 | 30.4 | 22.9 | 26.3 | 27.3 | — | — | |
| Llama-3.1-8B-InstructEvaluation Protocol=Single-step Reasoning Process Evaluator, Model Category=Instruction-tuned Models2025.03 | 24 | 15.5 | 9.7 | 10.1 | — | 14.8 | 0 |