Mathematical Reasoning on GSM8K (Accuracy @ N Tokens)
93.1Accuracy @ 128 TokensBiRM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BiRMBase Model=Qwen2.5-7B2025.03 | 93.1 | 93.3 | 93.2 | |
| PRMBase Model=Qwen2.5-7B2025.03 | 92.7 | 92.8 | 92.9 | |
| ER-PRMBase Model=Qwen2.5-7B2025.03 | 92.2 | 92.1 | 92.2 | |
| Math-ShepherdBase Model=Qwen2.5-7B2025.03 | 92.1 | 92.2 | 91.7 | |
| ORMBase Model=Qwen2.5-7B2025.03 | 92 | 91.6 | 91.3 | |
| PRMBase Model=Qwen2.5-3B2025.03 | 88.5 | 88.3 | 88 | |
| BiRMBase Model=Qwen2.5-3B2025.03 | 88.4 | 88.6 | 88.9 | |
| ORMBase Model=Qwen2.5-3B2025.03 | 88.1 | 88.1 | 88.1 | |
| Majority VoteBase Model=Qwen2.5-7B2025.03 | 88.1 | 88 | 87.8 | |
| ER-PRMBase Model=Qwen2.5-3B2025.03 | 88 | 88 | 87.7 | |
| Math-ShepherdBase Model=Qwen2.5-3B2025.03 | 87.3 | 87.2 | 87 | |
| BiRMBase Model=Llama3.1-8B2025.03 | 86.1 | 87.2 | 87.8 | |
| Majority VoteBase Model=Qwen2.5-3B2025.03 | 85.1 | 85 | 85.3 | |
| ER-PRMBase Model=Llama3.1-8B2025.03 | 84.8 | 85.3 | 85.8 | |
| Math-ShepherdBase Model=Llama3.1-8B2025.03 | 84.4 | 84.9 | 85.3 | |
| ORMBase Model=Llama3.1-8B2025.03 | 84.1 | 84.5 | 85 | |
| PRMBase Model=Llama3.1-8B2025.03 | 84.1 | 84.8 | 85.2 | |
| GreedyBase Model=Qwen2.5-7B2025.03 | 78.5 | 78.5 | 78.5 | |
| GreedyBase Model=Qwen2.5-3B2025.03 | 73.1 | 73.1 | 73.1 | |
| Majority VoteBase Model=Llama3.1-8B2025.03 | 72.1 | 72 | 72.3 | |
| GreedyBase Model=Llama3.1-8B2025.03 | 55.7 | 55.7 | 55.7 |