Arithmetic Reasoning on SVAMP (Accuracy)
96.01AccuracyGraph-GRPO
Evaluation results
| Method | Links | |
|---|---|---|
| Graph-GRPO2026.03 | 96.01 | |
| EIB-LEARNER2026.03 | 94.7 | |
| DUPModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 94.2 | |
| G-designer2026.03 | 93.1 | |
| Zero-shot PS+Model=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 92.6 | |
| AgentDropout2026.03 | 91.04 | |
| AgentPrune2026.03 | 90.58 | |
| Zero-shot CoTModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 90.4 | |
| Least-to-MostModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 90.3 | |
| LLM-Debate2026.03 | 90.11 | |
| Tree2026.03 | 88.91 | |
| SC (CoT)2026.03 | 88.59 | |
| BaseModel=Qwen1.5-32B, Infer Mode=Base2024.05 | 87.6 | |
| Random2026.03 | 87.54 | |
| CoT2026.03 | 87.36 | |
| Chain2026.03 | 87.17 | |
| Self-consistencyBackbone=GPT-3 Code-davinci-0022022.03 | 86.8 | |
| Self-consistencyBackbone=PaLM-540B2022.03 | 86.6 | |
| Vanilla2026.03 | 86.55 | |
| TAIAModel=Qwen1.5-32B, Infer Mode=TAIA2024.05 | 86.2 | |
| CoK + SCBase Model=gpt-3.5-turbo2023.06 | 86 | |
| ComplexCoT + SCBase Model=gpt-3.5-turbo2023.06 | 85.6 | |
| Manual CoT + SCBase Model=gpt-3.5-turbo2023.06 | 85 | |
| Complete2026.03 | 84.01 | |
| BaseModel=Qwen1.5-14B, Infer Mode=Base2024.05 | 83.6 | |
| TAIAModel=Qwen1.5-14B, Infer Mode=TAIA2024.05 | 83.1 | |
| Auto-CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Few-shot2024.04 | 82.9 | |
| LoRAModel=Qwen1.5-14B, Infer Mode=LoRA2024.05 | 82.8 | |
| LoRAModel=Qwen1.5-32B, Infer Mode=LoRA2024.05 | 82.6 | |
| DUPModel=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 82.5 | |
| RoPPerturbation=HW2025.06 | 82.4 | |
| CoKBase Model=gpt-3.5-turbo2023.06 | 81.4 | |
| Least-to-MostModel=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 80.9 | |
| Zero-shot PS+Model=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 80.7 | |
| CoTPerturbation=EC2025.06 | 80.3 | |
| RoPPerturbation=EC2025.06 | 79.7 | |
| Manual CoTBase Model=gpt-3.5-turbo2023.06 | 79.5 | |
| RoPPerturbation=WOO2025.06 | 79.5 | |
| StandPerturbation=No Dert.2025.06 | 79.4 | |
| Zero-shot CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 79.3 | |
| CoT-promptingBackbone=PaLM-540B2022.03 | 79 | |
| Manual-CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Few-shot2024.04 | 78.5 | |
| RoPPerturbation=SC2025.06 | 78.3 | |
| CoTPerturbation=WOO2025.06 | 78.2 | |
| APEPerturbation=HW2025.06 | 77.8 | |
| ComplexCoTBase Model=gpt-3.5-turbo2023.06 | 77.7 | |
| PromptAgentPerturbation=SC2025.06 | 77.4 | |
| StandPerturbation=WOO2025.06 | 76.9 | |
| PromptAgentPerturbation=WOO2025.06 | 76.8 | |
| RoPPerturbation=UIC2025.06 | 76.7 | |
| APEPerturbation=EC2025.06 | 76.5 | |
| PromptAgentPerturbation=EC2025.06 | 76.5 | |
| CoTPerturbation=SC2025.06 | 76.2 | |
| APEPerturbation=SC2025.06 | 76.2 | |
| CoT-promptingBackbone=GPT-3 Code-davinci-0022022.03 | 75.8 | |
| CoTPerturbation=HW2025.06 | 75.8 | |
| PromptAgentPerturbation=HW2025.06 | 75.5 | |
| StandPerturbation=EC2025.06 | 75 | |
| StandPerturbation=SC2025.06 | 74 | |
| APEPerturbation=WOO2025.06 | 72.7 | |
| CoKBase Model=text-davinci-0022023.06 | 69.9 | |
| Auto-CoTPrompting=Automatic Chain-of-Thought2022.10 | 69.5 | |
| Auto-CoTBase Model=text-davinci-0022023.06 | 69.5 | |
| Manual-CoTPrompting=Manual Chain-of-Thought2022.10 | 68.9 | |
| Manual CoTBase Model=text-davinci-0022023.06 | 68.9 | |
| TAIAModel=Qwen1.5-7B, Infer Mode=TAIA2024.05 | 67.2 | |
| StandPerturbation=HW2025.06 | 65.9 | |
| Few-ShotPrompting=Few-shot2022.10 | 65.7 | |
| Few-Shot SPBase Model=text-davinci-0022023.06 | 65.7 | |
| APEPerturbation=UIC2025.06 | 65.5 | |
| Zero-Shot-CoTPrompting=Zero-shot Chain-of-Thought2022.10 | 63.7 | |
| Zero-Shot CoTBase Model=text-davinci-0022023.06 | 62.1 | |
| Zero-ShotPrompting=Zero-shot2022.10 | 58.8 | |
| Zero-Shot SPBase Model=text-davinci-0022023.06 | 58.8 | |
| Pi et al. (2022)2022.03 | 57.4 | |
| Fine-tuning2023.06 | 57.4 | |
| LoRAModel=Qwen1.5-7B, Infer Mode=LoRA2024.05 | 57.1 | |
| CoTPerturbation=UIC2025.06 | 55.7 | |
| BaseModel=Qwen1.5-7B, Infer Mode=Base2024.05 | 54.9 | |
| Self-consistencyBackbone=GPT-3 Code-davinci-0012022.03 | 54.5 | |
| StandPerturbation=UIC2025.06 | 54.2 | |
| Self-consistencyBackbone=LaMDA-137B2022.03 | 53.3 | |
| PromptAgentPerturbation=UIC2025.06 | 49.2 | |
| CoT-promptingBackbone=GPT-3 Code-davinci-0012022.03 | 39.8 | |
| CoT-promptingBackbone=LaMDA-137B2022.03 | 38.9 | |
| Self-consistencyBackbone=UL2-20B2022.03 | 19.4 | |
| CoT-promptingBackbone=UL2-20B2022.03 | 12.6 |