Process-level verification on OlympiadBench ProcessBench (test)
8.2ErrorRLHFlow-PRM-Mistral-8B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RLHFlow-PRM-Mistral-8Btraining_data_source=automatically constructed datasets2025.05 | 8.2 | 43.1 | 13.8 | |
| RLHFlow-PRM-Deepseek-8Btraining_data_source=automatically constructed datasets2025.05 | 10.1 | 51 | 16.9 | |
| Math-Shepherd-PRM-7Btraining_data_source=automatically constructed datasets2025.05 | 15 | 71.1 | 24.8 | |
| Skywork-PRM-1.5Btraining_data_source=automatically constructed datasets2025.05 | 15.4 | 26 | 19.3 | |
| Skywork-PRM-7Btraining_data_source=automatically constructed datasets2025.05 | 17.9 | 31.9 | 22.9 | |
| SCOPE (Step Replacement)Step Compression Strategy=Step Replacement2025.05 | 23.8 | 69.3 | 35.4 | |
| EurusPRM-Stage1training_data_source=automatically constructed datasets2025.05 | 23.9 | 19.8 | 21.7 | |
| SCOPE (Step Skipping)Step Compression Strategy=Step Skipping2025.05 | 24.1 | 74.9 | 36.4 | |
| EurusPRM-Stage2training_data_source=automatically constructed datasets2025.05 | 25.7 | 18 | 21.2 | |
| SCOPE (w/o Code Translation)AST Normalization=true, Code Translation=false2025.05 | 32.7 | 54.9 | 41 | |
| SCOPE (w/o AST Normalization)AST Normalization=false, Code Translation=true2025.05 | 34.3 | 56.6 | 42.8 | |
| Qwen2.5-Math-7B-PRM800Ktraining_data_source=high-quality manually annotated process-level data (PRM800K)2025.05 | 35.7 | 87.3 | 50.7 | |
| SCOPEAST Normalization=true, Code Translation=true, training_data_source=automatically constructed datasets2025.05 | 35.7 | 59.3 | 44.6 |