Mathematical Reasoning on MATH (test) (Model-Specific Performance)
58.7Mistral-7B AccuracyOptimal
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| OptimalPRM=Llama3.1-Mistral-8B, Sample size=322025.10 | 58.7 | 65.8 | 74.4 | 65.7 | 61.9 | 65.3 | |
| OptimalPRM=Qwen-PRM-7B, Sample size=322025.10 | 58.5 | 67.9 | 75.4 | 67.6 | 63.1 | 66.5 | |
| OptimalPRM=Llama3.1-DS-8B, Sample size=322025.10 | 58.1 | 65.9 | 73.8 | 65.6 | 61.6 | 65 | |
| OptimalPRM=Skywork-PRM-1.5B, Sample size=322025.10 | 57.7 | 73.1 | 75.1 | 67.3 | 68.9 | 68.4 | |
| Linear WVPRM=Skywork-PRM-1.5B, Sample size=322025.10 | 55.8 | 70.3 | 72.2 | 66.2 | 65.5 | 66 | |
| OptimalPRM=Qwen-PRM800K-7B, Sample size=322025.10 | 55.6 | 64.5 | 71.7 | 59.3 | 57.2 | 61.7 | |
| Logit WVPRM=Llama3.1-Mistral-8B, Sample size=322025.10 | 55.6 | 62.9 | 70.7 | 62.8 | 60.7 | 62.6 | |
| Logit WVPRM=Skywork-PRM-1.5B, Sample size=322025.10 | 55.6 | 70.2 | 72 | 66 | 65.5 | 65.9 | |
| Logit WVPRM=Llama3.1-DS-8B, Sample size=322025.10 | 54.9 | 61.6 | 69.9 | 61.9 | 59.2 | 61.5 | |
| Vanilla WVPRM=Skywork-PRM-1.5B, Sample size=322025.10 | 54.9 | 69.4 | 71.6 | 60.3 | 61.5 | 63.5 | |
| Linear WVPRM=Llama3.1-Mistral-8B, Sample size=322025.10 | 54.7 | 62.2 | 70.1 | 62.3 | 60.2 | 61.9 | |
| Linear WVPRM=Llama3.1-DS-8B, Sample size=322025.10 | 54.7 | 61.5 | 69.9 | 60.7 | 58.9 | 61.1 | |
| Vanilla WVPRM=Qwen-PRM-7B, Sample size=322025.10 | 54.3 | 62.9 | 70.2 | 59.9 | 56.8 | 60.8 | |
| Logit WVPRM=Qwen-PRM-7B, Sample size=322025.10 | 54.3 | 63.5 | 70.5 | 65.1 | 62.9 | 63.3 | |
| Vanilla WVPRM=Llama3.1-DS-8B, Sample size=322025.10 | 54.3 | 61.7 | 69.6 | 57.1 | 55.7 | 59.7 | |
| BoNPRM=Llama3.1-Mistral-8B, Sample size=322025.10 | 54.2 | 58 | 68.7 | 56.9 | 53.5 | 58.3 | |
| BoNPRM=Skywork-PRM-1.5B, Sample size=322025.10 | 54.2 | 70.3 | 72.5 | 63.5 | 66.1 | 65.3 | |
| Linear WVPRM=Qwen-PRM-7B, Sample size=322025.10 | 54.1 | 63 | 70.3 | 64.7 | 62.6 | 62.9 | |
| Vanilla WVPRM=Llama3.1-Mistral-8B, Sample size=322025.10 | 53.9 | 62.2 | 69.4 | 56.9 | 54.8 | 59.4 | |
| KDE WVPRM=Llama3.1-Mistral-8B, Sample size=322025.10 | 53.9 | 62 | 68.8 | 58 | 58.1 | 60.2 | |
| KDE WVPRM=Skywork-PRM-1.5B, Sample size=322025.10 | 53.8 | 68.4 | 70.6 | 60.6 | 62.8 | 63.2 | |
| BoNPRM=Qwen-PRM-7B, Sample size=322025.10 | 53.7 | 62.6 | 70.4 | 62.5 | 59.8 | 61.8 | |
| KDE WVPRM=Llama3.1-DS-8B, Sample size=322025.10 | 53.3 | 61.4 | 68.7 | 58.3 | 56.9 | 59.7 | |
| Linear WVPRM=Qwen-PRM800K-7B, Sample size=322025.10 | 53 | 61.5 | 67.9 | 53.8 | 51.8 | 57.6 | |
| Logit WVPRM=Qwen-PRM800K-7B, Sample size=322025.10 | 53 | 61.5 | 67.7 | 53.9 | 51.9 | 57.6 | |
| KDE WVPRM=Qwen-PRM-7B, Sample size=322025.10 | 52.8 | 61.9 | 68.8 | 59.1 | 58.2 | 60.2 | |
| Vanilla WVPRM=Qwen-PRM800K-7B, Sample size=322025.10 | 52.5 | 61.4 | 67.7 | 53.7 | 51.7 | 57.4 | |
| KDE WVPRM=Qwen-PRM800K-7B, Sample size=322025.10 | 52.3 | 61.2 | 68 | 54 | 52 | 57.5 | |
| MVPRM=Qwen-PRM800K-7B, Sample size=322025.10 | 51.3 | 60.6 | 67.8 | 54.1 | 51.9 | 57.1 | |
| MVPRM=Qwen-PRM-7B, Sample size=322025.10 | 51.3 | 60.6 | 67.8 | 54.1 | 51.9 | 57.1 | |
| MVPRM=Llama3.1-Mistral-8B, Sample size=322025.10 | 51.3 | 60.6 | 67.8 | 54.1 | 51.9 | 57.1 | |
| BoNPRM=Llama3.1-DS-8B, Sample size=322025.10 | 51.3 | 59.5 | 68.8 | 59 | 55.9 | 58.9 | |
| MVPRM=Llama3.1-DS-8B, Sample size=322025.10 | 51.3 | 60.6 | 67.8 | 54.1 | 51.9 | 57.1 | |
| MVPRM=Skywork-PRM-1.5B, Sample size=322025.10 | 51.3 | 65.8 | 67.8 | 54.1 | 57.1 | 59.2 | |
| BoNPRM=Qwen-PRM800K-7B, Sample size=322025.10 | 49.5 | 57.3 | 65.3 | 45 | 46.1 | 52.7 |