Mathematical Reasoning on GSM-Hard
78Solve RateGPT-4o
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-4oPrompting=decl. PL-1S2025.12 | 78 | — | — | — | — | |
| GPT-4 (Code Interpreter)Decoding Strategy=Greedy2024.02 | 77.6 | — | — | — | — | |
| F-CoTModel=Qwen 3 14B2025.11 | 77.03 | — | 1,838 | — | — | |
| F-CoTModel=Qwen 3 32B2025.11 | 77.03 | — | 1,785 | — | — | |
| F-CoTModel=Qwen 3 4B2025.11 | 76.88 | — | 2,308 | — | — | |
| GPT-4oPrompting=PY-ZS2025.12 | 75.7 | — | — | — | — | |
| CoReModel=Qwen 3 14B2025.11 | 73.31 | — | 4,253 | — | — | |
| PSModel=Qwen 3 14B2025.11 | 72.86 | — | 4,285 | — | — | |
| GPT-4oPrompting=decl. PY-1S2025.12 | 72.6 | — | — | — | — | |
| CoReModel=Qwen 3 4B2025.11 | 71.95 | — | 5,509 | — | — | |
| 0-CoTModel=Qwen 3 14B2025.11 | 71.87 | — | 4,429 | — | — | |
| PSModel=Qwen 3 4B2025.11 | 71.49 | — | 5,318 | — | — | |
| GPT-4oPrompting=PL-ZS2025.12 | 71.3 | — | — | — | — | |
| CoReModel=Qwen 3 32B2025.11 | 71.27 | — | 4,387 | — | — | |
| PSModel=Qwen 3 32B2025.11 | 70.96 | — | 4,238 | — | — | |
| 0-CoTModel=Qwen 3 4B2025.11 | 70.36 | — | 5,674 | — | — | |
| 0-CoTModel=Qwen 3 32B2025.11 | 70.13 | — | 4,516 | — | — | |
| FEW-SHOTBackbone=QWEN3-8B2026.01 | 69.3 | — | — | — | — | |
| QWEN3-8BModel=QWEN3-8B, Method=FEW-SHOT2026.01 | 69.3 | — | — | — | — | |
| GPT-4oPrompting=CoT2025.12 | 69.1 | — | — | — | — | |
| Qwen2-Math-7B-InstructBackbone=Qwen2-Math-7B, Variant=Instruct2024.10 | 68.3 | 72.7 | — | — | — | |
| COMTBackbone=QWEN3-4B2026.01 | 67.9 | — | — | — | — | |
| COMT+CCRLBackbone=QWEN3-4B2026.01 | 67.9 | — | — | — | — | |
| ToRAModel Size=70B, Base Model=Llama-2, Decoding Strategy=Greedy2024.02 | 67.2 | — | — | — | — | |
| COMT+CCRLModel=QWEN3-4B, Method=COMT+CCRL2026.01 | 67 | — | — | — | — | |
| OpenMath-CodeLlamaModel Size=70B, Base Model=CodeLlama, Decoding Strategy=Greedy2024.02 | 66.6 | — | — | — | — | |
| COMTModel=QWEN3-4B, Method=COMT2026.01 | 66.6 | — | — | — | — | |
| CoReModel=Qwen 3 0.6B2025.11 | 66.34 | — | 4,854 | — | — | |
| Qwen2-Math-7B-ScaleQuestBackbone=Qwen2-Math-7B, Variant=ScaleQuest2024.10 | 66.3 | 72.5 | — | — | — | |
| FSLRBackbone=Qwen2.5-7B, Data Source=Self, Training Set=SVAMP2026.01 | 66.2 | — | — | — | — | |
| FSLRBackbone=Qwen3-4B, Data Source=LLaMA, Training Set=SVAMP2026.01 | 66 | — | — | — | — | |
| 0-CoTModel=Qwen 3 0.6B2025.11 | 65.88 | — | 5,235 | — | — | |
| OpenMath-Llama2Model Size=70B, Base Model=Llama-2, Decoding Strategy=Greedy2024.02 | 65.7 | — | — | — | — | |
| PSModel=Qwen 3 0.6B2025.11 | 65.43 | — | 4,989 | — | — | |
| COMT+CCRLModel=QWEN2.5-7B, Method=COMT+CCRL2026.01 | 65.3 | — | — | — | — | |
| F-CoTModel=Qwen 3 0.6B2025.11 | 64.9 | — | 2,122 | — | — | |
| FSLRBackbone=Qwen3-4B, Data Source=Self, Training Set=SVAMP2026.01 | 64.5 | — | — | — | — | |
| COMT+CCRLBackbone=QWEN3-8B2026.01 | 64.4 | — | — | — | — | |
| COMT+CCRLModel=QWEN3-8B, Method=COMT+CCRL2026.01 | 64.2 | — | — | — | — | |
| OpenMath-CodeLlamaModel Size=34B, Base Model=CodeLlama, Decoding Strategy=Greedy2024.02 | 64 | — | — | — | — | |
| COMTBackbone=QWEN3-8B2026.01 | 63.9 | — | — | — | — | |
| OpenMath-Mistral-7BModel Size=7B, Base Model=Mistral, Decoding Strategy=Greedy2024.02 | 63.7 | — | — | — | — | |
| ToRAModel Size=34B, Base Model=CodeLlama, Decoding Strategy=Greedy2024.02 | 63.7 | — | — | — | — | |
| FSLRBackbone=Qwen3-4B, Data Source=Qwen, Training Set=SVAMP2026.01 | 63.7 | — | — | — | — | |
| COMTModel=QWEN2.5-7B, Method=COMT2026.01 | 63.4 | — | — | — | — | |
| FEW-SHOTBackbone=QWEN2.5-7B2026.01 | 62.9 | — | — | — | — | |
| QWEN2.5-7BModel=QWEN2.5-7B, Method=FEW-SHOT2026.01 | 62.9 | — | — | — | — | |
| Base LLMBackbone=Qwen2.5-7B, Mode=Few-shot2026.01 | 62.9 | — | — | — | — | |
| COMTModel=QWEN3-8B, Method=COMT2026.01 | 62.5 | — | — | — | — | |
| ZERO-SHOTBackbone=QWEN3-4B2026.01 | 62.4 | — | — | — | — | |
| QWEN3-4BModel=QWEN3-4B, Method=ZERO-SHOT2026.01 | 62.4 | — | — | — | — | |
| Base LLMBackbone=Qwen3-4B, Mode=Zero-shot2026.01 | 62.4 | — | — | — | — | |
| CoT-SFTBackbone=Qwen3-4B, Data Source=Qwen, Training Set=SVAMP2026.01 | 62.2 | — | — | — | — | |
| OpenMath-CodeLlamaModel Size=13B, Base Model=CodeLlama, Decoding Strategy=Greedy2024.02 | 61.9 | — | — | — | — | |
| code-davinci-002Prompting=PoT2025.12 | 61.8 | — | — | — | — | |
| CoT-SFTBackbone=Qwen2.5-7B, Data Source=Self, Training Set=SVAMP2026.01 | 61.7 | — | — | — | — | |
| PALPrompting Strategy=Program-aided, Model=Codex2022.11 | 61.2 | — | — | — | — | |
| code-davinci-002Prompting=PAL2025.12 | 61.2 | — | — | — | — | |
| COT-SFT+RLModel=QWEN2.5-7B, Method=COT-SFT+RL2026.01 | 60.7 | — | — | — | — | |
| ToRAModel Size=13B, Base Model=CodeLlama, Decoding Strategy=Greedy2024.02 | 60.5 | — | — | — | — | |
| COT-SFT+RLBackbone=QWEN3-4B2026.01 | 60.5 | — | — | — | — | |
| OpenMath-CodeLlamaModel Size=7B, Base Model=CodeLlama, Decoding Strategy=Greedy2024.02 | 60.1 | — | — | — | — | |
| COMT+CCRLBackbone=QWEN2.5-7B2026.01 | 59.8 | — | — | — | — | |
| COT-SFT+RLModel=QWEN3-8B, Method=COT-SFT+RL2026.01 | 59.3 | — | — | — | — | |
| COT-SFT+RLBackbone=QWEN2.5-7B2026.01 | 59.2 | — | — | — | — | |
| CoT-SFTBackbone=Qwen2.5-7B, Data Source=Qwen, Training Set=SVAMP2026.01 | 58.8 | — | — | — | — | |
| CoT-SFTBackbone=Qwen3-4B, Data Source=LLaMA, Training Set=SVAMP2026.01 | 58.5 | — | — | — | — | |
| COMTBackbone=QWEN2.5-7B2026.01 | 58.3 | — | — | — | — | |
| COT-SFTBackbone=QWEN3-4B2026.01 | 58.1 | — | — | — | — | |
| FSLRBackbone=Qwen2.5-7B, Data Source=Qwen, Training Set=SVAMP2026.01 | 57.5 | — | — | — | — | |
| FSLRBackbone=Qwen2.5-7B, Data Source=LLaMA, Training Set=SVAMP2026.01 | 57.2 | — | — | — | — | |
| CoT-SFTBackbone=Qwen2.5-7B, Data Source=LLaMA, Training Set=SVAMP2026.01 | 57 | — | — | — | — | |
| COT-SFTBackbone=QWEN2.5-7B2026.01 | 56.2 | — | — | — | — | |
| ZERO-SHOTBackbone=DEEPSEEK-MATH2026.01 | 56.1 | — | — | — | — | |
| DeepSeek-MathModel=DeepSeek-Math, Method=ZERO-SHOT2026.01 | 56.1 | — | — | — | — | |
| DeepSeek-MathBackbone=DeepSeek-Math-7B, Mode=Zero-shot2026.01 | 56.1 | — | — | — | — | |
| ToRAModel Size=7B, Base Model=CodeLlama, Decoding Strategy=Greedy2024.02 | 56 | — | — | — | — | |
| FEW-SHOTBackbone=QWEN3-4B2026.01 | 56 | — | — | — | — | |
| QWEN3-4BModel=QWEN3-4B, Method=FEW-SHOT2026.01 | 56 | — | — | — | — | |
| Base LLMBackbone=Qwen3-4B, Mode=Few-shot2026.01 | 56 | — | — | — | — | |
| ZERO-SHOTBackbone=QWEN2.5-MATH2026.01 | 55.4 | — | — | — | — | |
| Qwen2.5-MathModel=Qwen2.5-Math, Method=ZERO-SHOT2026.01 | 55.4 | — | — | — | — | |
| Qwen2.5-MathBackbone=Qwen2.5-Math-7B, Mode=Zero-shot2026.01 | 55.4 | — | — | — | — | |
| CoT-SFTBackbone=Qwen3-4B, Data Source=Self, Training Set=SVAMP2026.01 | 55 | — | — | — | — | |
| COT-SFTModel=QWEN3-4B, Method=COT-SFT2026.01 | 54.8 | — | — | — | — | |
| DeepSeek-Coder-BaseSize=33B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 54.1 | — | — | — | — | |
| COT-SFTModel=QWEN3-8B, Method=COT-SFT2026.01 | 53.9 | — | — | — | — | |
| COT-SFT+RLModel=QWEN3-4B, Method=COT-SFT+RL2026.01 | 53.8 | — | — | — | — | |
| ZERO-SHOTBackbone=QWEN2.5-7B2026.01 | 53.4 | — | — | — | — | |
| QWEN2.5-7BModel=QWEN2.5-7B, Method=ZERO-SHOT2026.01 | 53.4 | — | — | — | — | |
| Base LLMBackbone=Qwen2.5-7B, Mode=Zero-shot2026.01 | 53.4 | — | — | — | — | |
| COT-SFTModel=QWEN2.5-7B, Method=COT-SFT2026.01 | 52.8 | — | — | — | — | |
| CodeLlama-BaseSize=34B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 51.8 | — | — | — | — | |
| COT-SFT+RLBackbone=QWEN3-8B2026.01 | 50.5 | — | — | — | — | |
| PPCVBackbone=Llama-3.1-8B-Instruct2026.02 | 49.73 | — | — | — | — | |
| ZERO-SHOTBackbone=QWEN3-8B2026.01 | 49.5 | — | — | — | — | |
| QWEN3-8BModel=QWEN3-8B, Method=ZERO-SHOT2026.01 | 49.5 | — | — | — | — | |
| COT-SFTBackbone=QWEN3-8B2026.01 | 46.5 | — | — | — | — | |
| COMT+CCRLModel=LLAMA3.1-8B, Method=COMT+CCRL2026.01 | 42.2 | — | — | — | — | |
| COMTModel=LLAMA3.1-8B, Method=COMT2026.01 | 41.8 | — | — | — | — |