Arithmetic Reasoning on MultiArith
100AccuracyPrevious SoTA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Previous SoTA2024.10 | 100 | — | — | |
| Self-consistencyBackbone=GPT-3 Code-davinci-0022022.03 | 100 | — | — | |
| DIVERSEModel=code-davinci-0022022.06 | 99.8 | — | — | |
| PALBackbone=Codex, Decoding strategy=Greedy2023.05 | 99.7 | — | — | |
| Automatic Model Selection with LLMsBackbone=Codex, Decoding strategy=Greedy2023.05 | 99.7 | — | — | |
| HyEvoBackbone=gpt-4o-mini2026.03 | 99.67 | — | — | |
| 8-shot CoTModel=PaLM-540B, Setting=Prev, # Paths=402023.05 | 99.3 | — | — | |
| Self-ConsistencyModel=PaLM 540B2022.06 | 99.3 | — | — | |
| Self-consistencyBackbone=PaLM-540B2022.03 | 99.3 | — | — | |
| CoK + SCBase Model=gpt-3.5-turbo2023.06 | 99.3 | — | — | |
| PALPrompting Strategy=Program-aided, Model=Codex2022.11 | 99.2 | — | — | |
| GoAgent2026.03 | 99.11 | — | — | |
| SIGMABase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=Full2026.06 | 99.11 | — | — | |
| Self-AgreementBackbone=GPT-3.5-turbo, Scenario=Question type unknown, answer format known2023.11 | 99.1 | — | — | |
| ST-EVOBackbone=gpt-oss-120b2026.02 | 99.05 | — | — | |
| Auto-CoTModel=PaLM-540B, Setting=0-shot, # Paths=142023.05 | 99 | — | — | |
| Automatic Model Selection with LLMsBackbone=GPT-4, Decoding strategy=Greedy2023.05 | 99 | — | — | |
| Teaching-Inspired Integrated Prompting FrameworkModel=GPT-42024.10 | 99 | — | — | |
| Auto-CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Few-shot2024.04 | 99 | — | — | |
| CoKBase Model=gpt-3.5-turbo2023.06 | 99 | — | — | |
| ARG-Designer2026.03 | 98.93 | — | — | |
| COSPModel=PaLM-540B, Setting=0-shot, # Paths=7+72023.05 | 98.8 | — | — | |
| Manual CoT + SCBase Model=gpt-3.5-turbo2023.06 | 98.8 | — | — | |
| MaASBackbone=gpt-4o-mini2026.03 | 98.8 | — | — | |
| CoTBackbone=ChatGPT, Decoding strategy=Greedy2023.05 | 98.7 | — | — | |
| Automatic Model Selection with LLMsBackbone=ChatGPT, Decoding strategy=Greedy2023.05 | 98.7 | — | — | |
| Meta-Reasoning ParadigmModel=ChatGPT (GPT-3.5-Turbo), Evaluation Protocol=Few-Shot2023.06 | 98.7 | — | — | |
| ARG-DesignerBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 98.67 | — | — | |
| Self-ConsistencyModel=code-davinci-0022022.06 | 98.6 | — | — | |
| CoTModel=GPT-42024.10 | 98.6 | — | — | |
| PALBackbone=GPT-4, Decoding strategy=Greedy2023.05 | 98.5 | — | — | |
| SIGMABase LLM=Qwen3-8B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=Full2026.06 | 98.39 | — | — | |
| SIGMABase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=Full2026.06 | 98.34 | — | — | |
| COPROModel=GPT-4o2026.05 | 98.33 | — | — | |
| MIPROModel=GPT-4o2026.05 | 98.33 | — | — | |
| BaselineLLM=GPT-4o2026.05 | 98.33 | — | — | |
| GEPALLM=GPT-4o2026.05 | 98.33 | — | — | |
| PromptAgentLLM=GPT-4o2026.05 | 98.33 | — | — | |
| COPROLLM=GPT-4o2026.05 | 98.33 | — | — | |
| MIPROLLM=GPT-4o2026.05 | 98.33 | — | — | |
| ProTeGiLLM=GPT-4o2026.05 | 98.28 | — | — | |
| ComplexCoT + SCBase Model=gpt-3.5-turbo2023.06 | 98.23 | — | — | |
| PSAO_LLMModel=GPT-4o2026.05 | 98.22 | — | — | |
| PSAO_LLMLLM=GPT-4o2026.05 | 98.22 | — | — | |
| CARDBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 98.21 | — | — | |
| Self-ConsistencyBackbone=GPT-3.5-turbo, Scenario=Question type unknown, answer format known2023.11 | 98.1 | — | — | |
| Zero-shot PS+Model=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 98.1 | — | — | |
| DUPModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 98.1 | — | — | |
| CoTBackbone=GPT-4, Decoding strategy=Greedy2023.05 | 98 | — | — | |
| Teaching-Inspired Integrated Prompting FrameworkModel=GPT-3.5-Turbo2024.10 | 98 | — | — | |
| DALAAgent Category=Multi-Agent2025.11 | 97.87 | — | — | |
| DUPModel=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 97.8 | — | — | |
| Zero-shot CoTModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 97.8 | — | — | |
| SafeSieveBackbone=gpt-oss-120b2026.02 | 97.8 | — | — | |
| AgentSquareBackbone=gpt-4o-mini2026.03 | 97.77 | — | — | |
| G-DesignerBackbone=gpt-oss-120b2026.02 | 97.6 | — | — | |
| ARG-DesignerBase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 97.56 | — | — | |
| CoReBackbone=GPT-J 12B2022.10 | 97.5 | — | — | |
| Mixed-Few-Shot CoTBackbone=GPT-3.5-turbo, Scenario=Question type unknown, answer format known2023.11 | 97.5 | — | — | |
| AgentVerseBackbone=gpt-4o-mini2026.03 | 97.5 | — | — | |
| MultiPersonaBackbone=gpt-4o-mini2026.03 | 97.49 | — | — | |
| ARG-DesignerBackbone=gpt-oss-120b2026.02 | 97.35 | — | — | |
| LLM-DebateBackbone=gpt-4o-mini2026.03 | 97.33 | — | — | |
| Manual CoTBase Model=gpt-3.5-turbo2023.06 | 97.3 | — | — | |
| LLM-BlenderBackbone=gpt-4o-mini2026.03 | 97.29 | — | — | |
| BaselineBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=Baseline2026.05 | 97.22 | — | — | |
| GEPABackbone Model=Gemini-2-Flash, Prompt Optimization Technique=GEPA2026.05 | 97.22 | — | — | |
| PromptAgentBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=PromptAgent2026.05 | 97.22 | — | — | |
| COPROBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=COPRO2026.05 | 97.22 | — | — | |
| PSAO_LLMBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=PSAO_LLM2026.05 | 97.22 | — | — | |
| DyLANBackbone=gpt-4o-mini2026.03 | 97.12 | — | — | |
| Least-to-MostModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 97.1 | — | — | |
| Graph-GRPO2026.03 | 97.07 | — | — | |
| MIPROBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=MIPRO2026.05 | 97.06 | — | — | |
| ARMADALoss Function=L_cosine, Teacher Model=Stable Diffusion, Student Model=LLaMA-7B, Zero-shot=true2026.03 | 97 | — | — | |
| CARDBase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 96.94 | — | — | |
| SCAgent Category=Single-Agent2025.11 | 96.88 | — | — | |
| G-DesignerBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 96.86 | — | — | |
| VanillaBackbone=gpt-4o-mini2026.03 | 96.85 | — | — | |
| EIB-LEARNER2026.03 | 96.83 | — | — | |
| EIB-LEARNER2026.03 | 96.83 | — | — | |
| GPTSwarmBackbone=gpt-4o-mini2026.03 | 96.79 | — | — | |
| G-DesignerBase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 96.78 | — | — | |
| Meta-Reasoning ParadigmModel=175B GPT-3 (text-davinci-003), Evaluation Protocol=Few-Shot2023.06 | 96.7 | — | — | |
| ComplexCoTBackbone=gpt-4o-mini2026.03 | 96.7 | — | — | |
| ATOMBackbone Model=Meta-Llama-3.1-8B-Instruct2026.05 | 96.61 | — | — | |
| SC (CoT×5)Backbone=gpt-4o-mini2026.03 | 96.58 | — | — | |
| ProTeGiBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=ProTeGi2026.05 | 96.56 | — | — | |
| G-designer2026.03 | 96.5 | — | — | |
| ARMADALoss Function=L_euclid, Teacher Model=Stable Diffusion, Student Model=LLaMA-7B, Zero-shot=true2026.03 | 96.5 | — | — | |
| G-Designer2026.03 | 96.5 | — | — | |
| CoTPerturbation=WOO2025.06 | 96.5 | — | — | |
| GPTSwarmBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Partial, Skill-based composition=No2026.06 | 96.5 | — | — | |
| AutoAgentsBackbone=gpt-4o-mini2026.03 | 96.42 | — | — | |
| PHPAgent Category=Multi-Agent2025.11 | 96.41 | — | — | |
| DIVERSEModel=text-davinci-0022022.06 | 96.4 | — | — | |
| LLM-Debate2026.03 | 96.36 | — | — | |
| LLM-Debate2026.03 | 96.36 | — | — | |
| CoTBackbone=gpt-4o-mini2026.03 | 96.31 | — | — | |
| PALBackbone=ChatGPT, Decoding strategy=Greedy2023.05 | 96.3 | — | — |