Process-level verification on MATH ProcessBench (test)
9.6Error RateRLHFlow-PRM-Mistral-8B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RLHFlow-PRM-Mistral-8Btraining_data_source=automatically constructed datasets2025.05 | 9.6 | 45.2 | 15.8 | |
| RLHFlow-PRM-Deepseek-8Btraining_data_source=automatically constructed datasets2025.05 | 10.9 | 51.9 | 16.9 | |
| Skywork-PRM-1.5Btraining_data_source=automatically constructed datasets2025.05 | 13.6 | 32.8 | 19.2 | |
| Skywork-PRM-7Btraining_data_source=automatically constructed datasets2025.05 | 14 | 41.9 | 21 | |
| Math-Shepherd-PRM-7Btraining_data_source=automatically constructed datasets2025.05 | 14.2 | 73 | 23.8 | |
| SCOPE (Step Skipping)Step Compression Strategy=Step Skipping2025.05 | 15.9 | 71 | 26 | |
| SCOPE (Step Replacement)Step Compression Strategy=Step Replacement2025.05 | 16.5 | 68.5 | 26.6 | |
| Math-Shepherd-PRM-7Btraining_data_source=automatically constructed datasets2025.05 | 18 | 82 | 29.5 | |
| RLHFlow-PRM-Deepseek-8Btraining_data_source=automatically constructed datasets2025.05 | 21.4 | 80 | 33.8 | |
| RLHFlow-PRM-Mistral-8Btraining_data_source=automatically constructed datasets2025.05 | 21.7 | 72.2 | 33.4 | |
| EurusPRM-Stage1training_data_source=automatically constructed datasets2025.05 | 21.9 | 24.5 | 23.1 | |
| EurusPRM-Stage2training_data_source=automatically constructed datasets2025.05 | 23.1 | 19.1 | 20.9 | |
| SCOPE (w/o Code Translation)AST Normalization=true, Code Translation=false2025.05 | 26.5 | 56.8 | 36.1 | |
| Qwen2.5-Math-7B-PRM800Ktraining_data_source=high-quality manually annotated process-level data (PRM800K)2025.05 | 29.8 | 86.1 | 44.3 | |
| SCOPE (w/o AST Normalization)AST Normalization=false, Code Translation=true2025.05 | 29.8 | 57.3 | 39.2 | |
| SCOPEAST Normalization=true, Code Translation=true, training_data_source=automatically constructed datasets2025.05 | 31.4 | 62.2 | 41.7 | |
| EurusPRM-Stage1training_data_source=automatically constructed datasets2025.05 | 33.3 | 38.2 | 35.6 | |
| SCOPE (Step Replacement)Step Compression Strategy=Step Replacement2025.05 | 35.5 | 83 | 49.8 | |
| EurusPRM-Stage2training_data_source=automatically constructed datasets2025.05 | 36.4 | 35 | 35.7 | |
| Skywork-PRM-1.5Btraining_data_source=automatically constructed datasets2025.05 | 37.9 | 65.2 | 48 | |
| SCOPE (Step Skipping)Step Compression Strategy=Step Skipping2025.05 | 38 | 84.5 | 52.5 | |
| Skywork-PRM-7Btraining_data_source=automatically constructed datasets2025.05 | 43.8 | 62.2 | 53.6 | |
| SCOPE (w/o Code Translation)AST Normalization=true, Code Translation=false2025.05 | 47 | 75.6 | 57.9 | |
| Qwen2.5-Math-7B-PRM800Ktraining_data_source=high-quality manually annotated process-level data (PRM800K)2025.05 | 48 | 90.1 | 62.6 | |
| SCOPEAST Normalization=true, Code Translation=true, training_data_source=automatically constructed datasets2025.05 | 50.8 | 74.9 | 60.6 | |
| SCOPE (w/o AST Normalization)AST Normalization=false, Code Translation=true2025.05 | 51.9 | 74.6 | 61.2 |