Arithmetic Reasoning on MAWPS
95.94AccuracyGPT-OSS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-OSSParameters=120B, N=1,7722026.06 | 95.94 | — | — | 1,700 | 25.46 | |
| Gemma4Parameters=31B, N=1,7722026.06 | 94.47 | — | — | 1,674 | 105.72 | |
| PaLM 540BPrompting=Chain of thought, External Calculator=true2022.01 | 93.5 | — | — | — | — | |
| CodexEngine=code-davinci-002, Prompting=Chain of thought, External Calculator=true2022.01 | 93.3 | — | — | — | — | |
| PaLM 540BPrompting=Chain of thought, External Calculator=false2022.01 | 93.3 | — | — | — | — | |
| CodexEngine=code-davinci-002, Prompting=Chain of thought, External Calculator=false2022.01 | 92.6 | — | — | — | — | |
| DeepSeek-R1Parameters=70B, N=1,7722026.06 | 88.43 | — | — | 1,567 | 432.68 | |
| Prior bestPrompting=N/A2022.01 | 88.4 | — | — | — | — | |
| GPT-3 175BEngine=text-davinci-002, Prompting=Chain of thought, External Calculator=true2022.01 | 87.5 | — | — | — | — | |
| GPT-3 175BEngine=text-davinci-002, Prompting=Chain of thought, External Calculator=false2022.01 | 87.1 | — | — | — | — | |
| PaLM 540BPrompting=Standard, External Calculator=false2022.01 | 79.2 | — | — | — | — | |
| CodexEngine=code-davinci-002, Prompting=Standard, External Calculator=false2022.01 | 78.7 | — | — | — | — | |
| GPT-3 175BEngine=text-davinci-002, Prompting=Standard, External Calculator=false2022.01 | 72.7 | — | — | — | — | |
| LaMDA 137BPrompting=Chain of thought, External Calculator=true2022.01 | 69.3 | — | — | — | — | |
| LaMDA 137BPrompting=Chain of thought, External Calculator=false2022.01 | 57.9 | — | — | — | — | |
| UL2 20BPrompting=CoT prompting, Calculator=True, Self-consistency=True2022.05 | 57.9 | — | — | — | — | |
| LaMDA 137BPrompting=Standard, External Calculator=false2022.01 | 43.2 | — | — | — | — | |
| UL2 20BPrompting=Chain of thought, External Calculator=true2022.01 | 42.7 | — | — | — | — | |
| UL2 20BPrompting=CoT prompting, Calculator=True2022.05 | 42.7 | — | — | — | — | |
| UL2 20BPrompting=Chain of thought, External Calculator=false2022.01 | 19.1 | — | — | — | — | |
| UL2 20BPrompting=CoT prompting2022.05 | 19.1 | — | — | — | — | |
| UL2 20BPrompting=Standard, External Calculator=false2022.01 | 16.6 | — | — | — | — | |
| UL2 20BPrompting=standard prompting2022.05 | 16.6 | — | — | — | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=SAF2025.07 | — | 78.6 | 84.5 | — | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=MAF2025.07 | — | 81.1 | 80.7 | — | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=SSF2025.07 | — | 80.7 | 79.8 | — | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=MSF2025.07 | — | 79.4 | 80.7 | — | — | |
| CRFTBackbone=LLaMA-3-8B, Identify=SAF2025.07 | — | 89.9 | 88.2 | — | — | |
| CRFTBackbone=LLaMA-3-8B, Identify=MAF2025.07 | — | 90.8 | 90.8 | — | — | |
| CRFTBackbone=LLaMA-3-8B, Identify=SSF2025.07 | — | 86.6 | 86.6 | — | — | |
| CRFTBackbone=LLaMA-3-8B, Identify=MSF2025.07 | — | 87 | 87.4 | — | — | |
| CRFTBackbone=Mistral-7B, Identify=SAF2025.07 | — | 87 | 85.7 | — | — | |
| CRFTBackbone=Mistral-7B, Identify=MAF2025.07 | — | 84.9 | 85.3 | — | — | |
| CRFTBackbone=Mistral-7B, Identify=SSF2025.07 | — | 85.3 | 84.5 | — | — | |
| CRFTBackbone=Mistral-7B, Identify=MSF2025.07 | — | 87.4 | 85.3 | — | — | |
| ReFTBackbone=LLaMA-2-7B, Identify=p7+s72025.07 | — | 80.7 | 80.7 | — | — | |
| ReFTBackbone=LLaMA-3-8B, Identify=p7+s72025.07 | — | 87 | 87 | — | — | |
| ReFTBackbone=Mistral-7B, Identify=p7+s72025.07 | — | 84.9 | 84.9 | — | — |