Process-level verification on GSM8K ProcessBench (test)
24.2Error RateRLHFlow-PRM-Deepseek-8B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RLHFlow-PRM-Deepseek-8Btraining_data_source=automatically constructed datasets2025.05 | 24.2 | 98.4 | 38.8 | |
| Math-Shepherd-PRM-7Btraining_data_source=automatically constructed datasets2025.05 | 32.4 | 91.7 | 47.9 | |
| RLHFlow-PRM-Mistral-8Btraining_data_source=automatically constructed datasets2025.05 | 33.8 | 99 | 50.4 | |
| EurusPRM-Stage1training_data_source=automatically constructed datasets2025.05 | 46.9 | 42 | 44.3 | |
| Skywork-PRM-1.5Btraining_data_source=automatically constructed datasets2025.05 | 50.2 | 71.5 | 59 | |
| EurusPRM-Stage2training_data_source=automatically constructed datasets2025.05 | 51.2 | 44 | 47.3 | |
| Qwen2.5-Math-7B-PRM800Ktraining_data_source=high-quality manually annotated process-level data (PRM800K)2025.05 | 53.1 | 95.3 | 68.2 | |
| SCOPE (Step Replacement)Step Compression Strategy=Step Replacement2025.05 | 53.6 | 87.6 | 66.5 | |
| SCOPE (w/o Code Translation)AST Normalization=true, Code Translation=false2025.05 | 56.5 | 83.9 | 67.6 | |
| SCOPE (Step Skipping)Step Compression Strategy=Step Skipping2025.05 | 56.5 | 90.2 | 69.5 | |
| SCOPEAST Normalization=true, Code Translation=true, training_data_source=automatically constructed datasets2025.05 | 59.9 | 86 | 70.6 | |
| SCOPE (w/o AST Normalization)AST Normalization=false, Code Translation=true2025.05 | 61.4 | 85.5 | 71.4 | |
| Skywork-PRM-7Btraining_data_source=automatically constructed datasets2025.05 | 61.8 | 82.9 | 70.8 |