Mathematical Reasoning on GSM
94AccuracyQwenPRM
Evaluation results
| Method | Links | |
|---|---|---|
| QwenPRMModel Size=7B2026.03 | 94 | |
| MCNIGModel Size=8B2026.03 | 93.8 | |
| ORMModel Size=8B2026.03 | 93.7 | |
| IGModel Size=8B2026.03 | 93.3 | |
| IDPOModel=Qwen3-8B2025.05 | 93 | |
| SAI-DPOModel=Qwen3-8B2025.05 | 92.9 | |
| GPT-4Shots=8-shot, Method=CoT2023.06 | 92.5 | |
| Base ModelModel=Qwen3-8B2025.05 | 92.1 | |
| OVM2026.03 | 92 | |
| SAI-DPOModel=Qwen2.5-7B-Math-SFT2025.05 | 91.7 | |
| IDPOModel=Qwen2.5-7B-Math-SFT2025.05 | 91.3 | |
| Majority voting2026.03 | 91.2 | |
| Granite PRM v22026.03 | 90.9 | |
| Base ModelModel=Qwen2.5-7B-Math-SFT2025.05 | 90.8 | |
| ImplicitPRM2026.03 | 89.9 | |
| MathShepherd2026.03 | 89.5 | |
| SAI-DPOModel=Qwen2.5-7B-Math-Base2025.05 | 89.1 | |
| IDPOModel=Qwen2.5-7B-Math-Base2025.05 | 88.4 | |
| SATLMLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 84.8 | |
| PROGLMLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 82.4 | |
| SAI-DPOModel=Llama3.1-8B-Instruct2025.05 | 77.4 | |
| COTLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 77.3 | |
| IDPOModel=Llama3.1-8B-Instruct2025.05 | 76.9 | |
| ChatGPTShots=8-shot, Method=CoT2023.06 | 76 | |
| DPSLLM Backbone=GPT-3.5 Turbo2025.02 | 75.5 | |
| OPROLLM Backbone=GPT-3.5 Turbo2025.02 | 73.4 | |
| Single sampling2026.03 | 73.3 | |
| DPSLLM Backbone=Gemini-1.5 Flash2025.02 | 72.9 | |
| PROGLMLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 72.7 | |
| APELLM Backbone=GPT-3.5 Turbo2025.02 | 72.1 | |
| SATLMLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 71.8 | |
| DPSLLM Backbone=Llama-3.1 70B2025.02 | 71.8 | |
| OPROLLM Backbone=Gemini-1.5 Flash2025.02 | 70.4 | |
| APELLM Backbone=Gemini-1.5 Flash2025.02 | 69.8 | |
| OPROLLM Backbone=Llama-3.1 70B2025.02 | 69.3 | |
| Base ModelModel=Llama3.1-8B-Instruct2025.05 | 69.1 | |
| APELLM Backbone=Llama-3.1 70B2025.02 | 67.5 | |
| Base ModelModel=Qwen2.5-7B-Math-Base2025.05 | 66.7 | |
| COTLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 62.7 | |
| Human mix. 65BShots=8-shot, Method=CoT2023.06 | 60 | |
| ShareGPT 65BShots=8-shot, Method=CoT2023.06 | 59 | |
| TÜLU 65BShots=8-shot, Method=CoT2023.06 | 59 | |
| DenseMixerBackbone=Qwen1.5-MoE-A2.7B, Top-k=42026.06 | 54.97 | |
| Frozen RouterBackbone=Qwen1.5-MoE-A2.7B, Top-k=42026.06 | 53.37 | |
| ConventionalBackbone=Qwen1.5-MoE-A2.7B, Top-k=42026.06 | 53.3 | |
| ProbMoEBackbone=Qwen1.5-MoE-A2.7B, Top-k=42026.06 | 53.29 | |
| TÜLU 30BShots=8-shot, Method=CoT2023.06 | 53 | |
| TÜLU-1.1 13BShots=8-shot, Method=CoT2023.06 | 53 | |
| DefaultMoEBackbone=Qwen1.5-MoE-A2.7B, Top-k=42026.06 | 51 | |
| ProbMoEBackbone=OLMoE-1B-7B, Top-k=82026.06 | 50.19 | |
| LLaMa 65BShots=8-shot, Method=CoT2023.06 | 50 | |
| DenseMixerBackbone=OLMoE-1B-7B, Top-k=82026.06 | 47 | |
| ReMoEBackbone=Qwen1.5-MoE-A2.7B, Top-k=42026.06 | 46.3 | |
| ConventionalBackbone=OLMoE-1B-7B, Top-k=82026.06 | 45.94 | |
| Frozen RouterBackbone=OLMoE-1B-7B, Top-k=82026.06 | 44.88 | |
| TÜLU 13BShots=8-shot, Method=CoT2023.06 | 40.5 | |
| Base ModelBackbone=Qwen1.5-MoE-A2.7B, Top-k=42026.06 | 38.69 | |
| TÜLU-1.1 7BShots=8-shot, Method=CoT2023.06 | 37 | |
| LLaMa 30BShots=8-shot, Method=CoT2023.06 | 36 | |
| N/ALLM Backbone=Gemini-1.5 Flash2025.02 | 29.9 | |
| N/ALLM Backbone=GPT-3.5 Turbo2025.02 | 28.5 | |
| N/ALLM Backbone=Llama-3.1 70B2025.02 | 28.1 | |
| LLaMa-2 13BShots=8-shot, Method=CoT2023.06 | 25 | |
| TÜLU 7BShots=8-shot, Method=CoT2023.06 | 25 | |
| STANDARDLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 22.2 | |
| Base ModelBackbone=OLMoE-1B-7B, Top-k=82026.06 | 15.85 | |
| LLaMa 13BShots=8-shot, Method=CoT2023.06 | 14.5 | |
| LLaMa-2 7BShots=8-shot, Method=CoT2023.06 | 12 | |
| LLaMa 7BShots=8-shot, Method=CoT2023.06 | 10 | |
| SparseMixerBackbone=Qwen1.5-MoE-A2.7B, Top-k=42026.06 | 1.3 |