Multilingual Mathematical Reasoning on MGSM (test)
93.4AccuracyQuaSAR
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| QuaSARModel=GPT-4o2025.02 | 93.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoTModel=GPT-4o2025.02 | 91 | — | — | — | — | — | — | — | — | — | — | — | — | |
| baselineModel=GPT-4o2025.02 | 90.5 | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeLlama-7BPrompting=POT Parallel, Inference Strategy=Soft-SC (ICE-Score)2025.02 | 75.6 | 68.8 | — | — | — | — | 77.6 | — | — | — | 79.2 | — | — | |
| MSD (Off-Policy)Base Model=Qwen-3-8B2026.05 | 75.53 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MSD (On-Policy)Base Model=Qwen-3-8B2026.05 | 74.04 | — | — | — | — | — | — | — | — | — | — | — | — | |
| SFTBase Model=Qwen-3-8B2026.05 | 73.42 | — | — | — | — | — | — | — | — | — | — | — | — | |
| PASMRBase Model=Deepseek-math-7b-instruct2026.01 | 72.4 | 60.8 | 69.6 | 49.2 | 69.2 | 76.4 | 78.8 | 76.4 | 77.2 | 79.6 | 86.4 | 59.9 | — | |
| MSFT w. Gold Answer RLBase Model=Deepseek-math-7b-instruct2026.01 | 71.7 | 59.2 | 70.4 | 44 | 73.6 | 74 | 78.8 | 73.6 | 80.4 | 78 | 85.2 | 57.9 | — | |
| PASMR w. OOD dataBase Model=Deepseek-math-7b-instruct2026.01 | 71.6 | 60.4 | 68.4 | 47.2 | 66.8 | 76.4 | 78.4 | 76 | 77.6 | 80.4 | 84.8 | 58.7 | — | |
| MAPOBase Model=Deepseek-math-7b-instruct2026.01 | 71.3 | 62.8 | 67.6 | 44 | 70 | 75.2 | 77.2 | 74.8 | 77.6 | 78.8 | 85.2 | 58.1 | — | |
| Llama2-7BPrompting=POT Parallel, Inference Strategy=Soft-SC (ICE-Score)2025.02 | 71.2 | 65.6 | — | — | — | — | 72 | — | — | — | 77.6 | — | — | |
| MSFTBase Model=Deepseek-math-7b-instruct2026.01 | 69.2 | 56 | 65.6 | 40.4 | 68 | 72.4 | 74.8 | 75.6 | 76 | 77.2 | 86 | 54 | — | |
| SimPoBase Model=Qwen-3-8B2026.05 | 67.85 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ORPOBase Model=Qwen-3-8B2026.05 | 67.35 | — | — | — | — | — | — | — | — | — | — | — | — | |
| PASMR w. OOD dataBase Model=Llama-3-8B-Instruct2026.01 | 67 | 60 | 64 | 55.6 | 59.6 | 66 | 72.4 | 68.8 | 70.8 | 73.2 | 79.6 | 59.9 | — | |
| PASMRBase Model=Llama-3-8B-Instruct2026.01 | 66.9 | 59.6 | 59.6 | 57.2 | 57.6 | 66.8 | 73.2 | 67.2 | 72.8 | 74.4 | 80.4 | 58.8 | — | |
| MSFT w. Gold Answer RLBase Model=Llama-3-8B-Instruct2026.01 | 66 | 55.2 | 68 | 52 | 58.4 | 64.8 | 70.4 | 68.4 | 70 | 73.2 | 79.6 | 58.4 | — | |
| rDPOBase Model=Qwen-3-8B2026.05 | 65.89 | — | — | — | — | — | — | — | — | — | — | — | — | |
| KTOBase Model=Qwen-3-8B2026.05 | 65.89 | — | — | — | — | — | — | — | — | — | — | — | — | |
| R-DPOBase Model=Qwen-3-8B2026.05 | 65.49 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPOBase Model=Qwen-3-8B2026.05 | 65.35 | — | — | — | — | — | — | — | — | — | — | — | — | |
| RawBase Model=Qwen-3-8B2026.05 | 64.87 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MPOBase Model=Qwen-3-8B2026.05 | 64.62 | — | — | — | — | — | — | — | — | — | — | — | — | |
| PipelineBase Model=Llama-3-8B-Instruct2026.01 | 64.5 | 58.6 | 60.6 | 42.6 | 61.6 | 61.2 | 69.4 | 69.2 | 70.8 | 73.2 | 78 | 53.9 | — | |
| PolyRefuseBase Model=Qwen-3-8B2026.05 | 64.29 | — | — | — | — | — | — | — | — | — | — | — | — | |
| RawBase Model=Qwen-2.5-7B-Instruct2026.05 | 63.89 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MSD (On-Policy)Base Model=Qwen-2.5-7B-Instruct2026.05 | 63.89 | — | — | — | — | — | — | — | — | — | — | — | — | |
| rDPOBase Model=Qwen-2.5-7B-Instruct2026.05 | 63.16 | — | — | — | — | — | — | — | — | — | — | — | — | |
| KTOBase Model=Qwen-2.5-7B-Instruct2026.05 | 63.05 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MPOBase Model=Qwen-2.5-7B-Instruct2026.05 | 62.95 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPOBase Model=Qwen-2.5-7B-Instruct2026.05 | 62.91 | — | — | — | — | — | — | — | — | — | — | — | — | |
| R-DPOBase Model=Qwen-2.5-7B-Instruct2026.05 | 62.84 | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeLlama-7BPrompting=POT Parallel, Inference Strategy=Self-Consistency (SC)2025.02 | 62.8 | 53.6 | — | — | — | — | 66.4 | — | — | — | 68.8 | — | — | |
| SDRRLBase Model=Qwen-3-8B2026.05 | 62.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Deepseek-math-7b-instructBase Model=Deepseek-math-7b-instruct2026.01 | 61.8 | 46.4 | 58 | 13.2 | 64 | 72.8 | 69.2 | 67.6 | 72.4 | 70 | 84 | 39.2 | — | |
| CodeLlama-7BPrompting=Without Comments, Inference Strategy=Soft-SC (ICE-Score)2025.02 | 61.1 | 33.6 | — | — | — | — | 71.2 | — | — | — | 75.7 | — | — | |
| MSD (Off-Policy)Base Model=Qwen-2.5-7B-Instruct2026.05 | 60.44 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MAPOBase Model=Llama-3-8B-Instruct2026.01 | 60.2 | 49.2 | 60.4 | 46.4 | 50 | 61.6 | 63.6 | 65.2 | 62 | 68.4 | 74.8 | 52 | — | |
| MCOTBase Model=Deepseek-math-7b-instruct2026.01 | 60 | 46 | 56.4 | 13.6 | 63.2 | 71.6 | 65.2 | 64.4 | 72.4 | 69.2 | 78.4 | 38.6 | — | |
| ORPOBase Model=Qwen-2.5-7B-Instruct2026.05 | 59.89 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MSFTBase Model=Llama-3-8B-Instruct2026.01 | 59 | 51.6 | 57.6 | 46.4 | 49.6 | 59.2 | 62.4 | 62.8 | 63.2 | 66.4 | 71.2 | 51.9 | — | |
| PolyRefuseBase Model=Qwen-2.5-7B-Instruct2026.05 | 58.69 | — | — | — | — | — | — | — | — | — | — | — | — | |
| SDRRLBase Model=Qwen-2.5-7B-Instruct2026.05 | 58.58 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MSD (On-Policy)Base Model=LLaMA-3-8B-Instruct2026.05 | 58.11 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCOTBase Model=Llama-3-8B-Instruct2026.01 | 58 | 49.6 | 55.6 | 44.4 | 53.6 | 56.8 | 66.4 | 54.8 | 65.2 | 62.8 | 70.8 | 49.8 | — | |
| Llama-3-8B-InstructBase Model=Llama-3-8B-Instruct2026.01 | 57.8 | 48.4 | 55.6 | 32 | 49.6 | 56.8 | 61.6 | 63.2 | 64.8 | 67.2 | 78.4 | 45.3 | — | |
| Llama2-7BPrompting=POT Parallel, Inference Strategy=Self-Consistency (SC)2025.02 | 57.2 | 47.6 | — | — | — | — | 58 | — | — | — | 64.8 | — | — | |
| QAlign - MonoReasonSystem Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 57.1 | 38.4 | 49.6 | 46 | 52.4 | 59.2 | 62 | 62.4 | 64.4 | 67.2 | 69.2 | — | — | |
| Llama2-7BPrompting=Without Comments, Inference Strategy=Soft-SC (ICE-Score)2025.02 | 56.6 | 33.6 | — | — | — | — | 69.2 | — | — | — | 76.8 | — | — | |
| RawBase Model=LLaMA-3-8B-Instruct2026.05 | 56.04 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MSD (Off-Policy)Base Model=LLaMA-3-8B-Instruct2026.05 | 55.89 | — | — | — | — | — | — | — | — | — | — | — | — | |
| PolyRefuseBase Model=LLaMA-3-8B-Instruct2026.05 | 55.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| SimPoBase Model=Qwen-2.5-7B-Instruct2026.05 | 55.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MPOBase Model=LLaMA-3-8B-Instruct2026.05 | 54.25 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPOBase Model=LLaMA-3-8B-Instruct2026.05 | 53.13 | — | — | — | — | — | — | — | — | — | — | — | — | |
| SFTBase Model=Qwen-2.5-7B-Instruct2026.05 | 52.95 | — | — | — | — | — | — | — | — | — | — | — | — | |
| R-DPOBase Model=LLaMA-3-8B-Instruct2026.05 | 52.69 | — | — | — | — | — | — | — | — | — | — | — | — | |
| PASMRBase Model=Mistral-7B-Instruct2026.01 | 52.5 | 42.4 | 45.6 | 45.2 | 46.4 | 52.8 | 57.6 | 56.4 | 54.4 | 60.8 | 63.2 | 44.4 | — | |
| KTOBase Model=LLaMA-3-8B-Instruct2026.05 | 52.36 | — | — | — | — | — | — | — | — | — | — | — | — | |
| rDPOBase Model=LLaMA-3-8B-Instruct2026.05 | 51.89 | — | — | — | — | — | — | — | — | — | — | — | — | |
| QAlign - MonoReasonSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 49.6 | 32.4 | 39.6 | 40.4 | 44 | 48.4 | 54.8 | 56.8 | 52.4 | 59.6 | 68 | — | — | |
| CodeLlama-7BPrompting=POT Parallel2025.02 | 49 | 35.6 | — | — | — | — | 52.8 | — | — | — | 61.9 | — | — | |
| MSFT w. Gold Answer RLBase Model=Mistral-7B-Instruct2026.01 | 48.8 | 36.8 | 44 | 40.4 | 43.2 | 48.8 | 50 | 52.8 | 53.6 | 56.4 | 61.6 | 40.4 | — | |
| MultiReasonSystem Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 48.1 | 37.6 | 42.2 | 44 | 43.2 | 53.6 | 47.6 | 54 | 48 | 54.8 | 56.4 | — | — | |
| PASMR w. OOD dataBase Model=Mistral-7B-Instruct2026.01 | 48.1 | 39.2 | 44.4 | 37.2 | 40.8 | 53.2 | 51.6 | 50 | 53.2 | 55.2 | 56 | 40.3 | — | |
| SimPoBase Model=LLaMA-3-8B-Instruct2026.05 | 47.78 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ORPOBase Model=LLaMA-3-8B-Instruct2026.05 | 47.02 | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeLlama-7BPrompting=Without Comments, Inference Strategy=Self-Consistency (SC)2025.02 | 46.7 | 17.2 | — | — | — | — | 57.2 | — | — | — | 69.6 | — | — | |
| MathOctopus+System Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 45.8 | 35.2 | 46.8 | 42.8 | 43.2 | 48.8 | 44.4 | 48.4 | 47.6 | 48 | 53.2 | — | — | |
| Llama2-7BPrompting=POT Parallel2025.02 | 44.6 | 30.8 | — | — | — | — | 47.2 | — | — | — | 56 | — | — | |
| MetaMathSystem Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 44.4 | 11.6 | 6.4 | 7.6 | 42.8 | 49.2 | 64.8 | 65.2 | 63.6 | 65.2 | 67.2 | — | — | |
| MonoReasonSystem Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 43.9 | 12.4 | 11.2 | 6.4 | 42 | 46 | 64 | 62.4 | 61.6 | 64.8 | 68.4 | — | — | |
| SDRRLBase Model=LLaMA-3-8B-Instruct2026.05 | 42.65 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MSFTBase Model=Mistral-7B-Instruct2026.01 | 41.6 | 32 | 41.2 | 31.2 | 35.2 | 42 | 44 | 45.6 | 42.8 | 49.2 | 52.8 | 34.8 | — | |
| MathOctopusSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 40 | 28.8 | 34.4 | 39.2 | 36 | 38.4 | 44.8 | 43.6 | 39.6 | 42.4 | 52.4 | — | — | |
| Llama2-7BPrompting=Without Comments, Inference Strategy=Self-Consistency (SC)2025.02 | 39.2 | 15.2 | — | — | — | — | 51.6 | — | — | — | 65.2 | — | — | |
| MultiReasonSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 39.1 | 26.8 | 36 | 36.8 | 33.2 | 42.4 | 42.8 | 40.8 | 42.4 | 42.8 | 47.2 | — | — | |
| CodeLlama-7BPrompting=Without Comments2025.02 | 38.6 | 11.2 | — | — | — | — | 48.4 | — | — | — | 58.8 | — | — | |
| MonoReasonSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 38.4 | 7.6 | 5.6 | 5.2 | 34 | 45.2 | 54 | 56.8 | 51.6 | 58.8 | 65.5 | — | — | |
| MetaMathSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 37 | 6.4 | 4 | 3.2 | 39.2 | 38.8 | 56.8 | 52.8 | 47.2 | 58 | 63.2 | — | — | |
| PipelineBase Model=Deepseek-math-7b-instruct2026.01 | 33.9 | 16.6 | 29.6 | 5.6 | 32.6 | 25.2 | 44.8 | 49.8 | 51.8 | 45.6 | 37.6 | 17.2 | — | |
| Llama2-7BPrompting=Without Comments2025.02 | 31.6 | 12 | — | — | — | — | 40.4 | — | — | — | 58 | — | — | |
| MCOTBase Model=Mistral-7B-Instruct2026.01 | 31 | 19.6 | 24.8 | 7.6 | 26.8 | 34 | 36.4 | 32.4 | 39.2 | 41.6 | 48 | 17.3 | — | |
| MAPOBase Model=Mistral-7B-Instruct2026.01 | 30.8 | 23.6 | 25.2 | 23.2 | 29.2 | 31.2 | 31.2 | 35.6 | 34 | 35.6 | 39.2 | 24 | — | |
| SFTSystem Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 29.7 | 6 | 6.8 | 7.6 | 25.2 | 32.8 | 42.8 | 40.8 | 39.2 | 45.2 | 50.4 | — | — | |
| RFTSystem Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 29.5 | 3.2 | 4.4 | 3.6 | 26.4 | 33.6 | 38.4 | 44.8 | 41.6 | 46.8 | 52 | — | — | |
| MAmmoTHSystem Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 28.9 | 3.6 | 5.2 | 1.6 | 19.2 | 31.2 | 45.6 | 39.6 | 36.8 | 50 | 56.4 | — | — | |
| WizardMathSystem Size=13B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 28.3 | 6.4 | 5.6 | 5.6 | 22 | 28 | 40.4 | 42 | 34.4 | 45.6 | 52.8 | — | — | |
| PipelineBase Model=Mistral-7B-Instruct2026.01 | 27.7 | 18.8 | 16 | 3.8 | 27 | 29 | 36.8 | 33.4 | 35.2 | 35.6 | 41.6 | 12.8 | — | |
| Mistral-7B-InstructBase Model=Mistral-7B-Instruct2026.01 | 27 | 16 | 12 | 6.8 | 20.4 | 32.8 | 33.2 | 34.4 | 35.2 | 37.6 | 41.2 | 11.6 | — | |
| WizardMathSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 23 | 2 | 4 | 3.4 | 24 | 22.4 | 30.4 | 30.4 | 30.8 | 34.8 | 47.6 | — | — | |
| SFTSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 22.6 | 3.2 | 4.8 | 5.2 | 15.2 | 22.4 | 37.2 | 34.4 | 28 | 32.4 | 43.2 | — | — | |
| MAmmoTHSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 21.3 | 3.6 | 2.4 | 1.8 | 10.8 | 17.2 | 33.2 | 32.8 | 26 | 32.4 | 49.6 | — | — | |
| SFTBase Model=LLaMA-3-8B-Instruct2026.05 | 20.84 | — | — | — | — | — | — | — | — | — | — | — | — | |
| RFTSystem Size=7B, Evaluation Mode=Zero-shot, Decoding=Greedy2024.01 | 20.6 | 2.4 | 2 | 2.8 | 6.8 | 16.8 | 33.6 | 34 | 29.2 | 34 | 44.8 | — | — | |
| MSD (On-Policy)Base Model=LLaMA-2-7B-Chat2026.05 | 12.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MPOBase Model=LLaMA-2-7B-Chat2026.05 | 11.92 | — | — | — | — | — | — | — | — | — | — | — | — | |
| SDRRLBase Model=LLaMA-2-7B-Chat2026.05 | 11.75 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ORPOBase Model=LLaMA-2-7B-Chat2026.05 | 11.4 | — | — | — | — | — | — | — | — | — | — | — | — |