Mathematical Reasoning on AQuA (Accuracy)
93.55AccuracyProTeGi
Evaluation Results
| Method | Links | |
|---|---|---|
| ProTeGiBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=ProTeGi2026.05 | 93.55 | |
| PSAO_LLMBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=PSAO_LLM2026.05 | 93.42 | |
| COPROBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=COPRO2026.05 | 92.76 | |
| PSAO_LLMModel=GPT-4o2026.05 | 92.65 | |
| PSAO_LLMLLM=GPT-4o2026.05 | 92.65 | |
| PromptAgentBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=PromptAgent2026.05 | 92.24 | |
| CLoTBackbone=deepseek-v32026.04 | 91.8 | |
| BaselineBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=Baseline2026.05 | 91.45 | |
| GEPABackbone Model=Gemini-2-Flash, Prompt Optimization Technique=GEPA2026.05 | 90.13 | |
| MIPROBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=MIPRO2026.05 | 88.95 | |
| AgentReviveBase model=Deepseek-V3-671B-Instruct, Var. NS=true, Flex. State=true2026.05 | 88.85 | |
| AgentDropoutBase model=Deepseek-V3-671B-Instruct, Var. NS=true, Flex. State=false2026.05 | 88.33 | |
| C-CoTBackbone=deepseek-v32026.04 | 88.1 | |
| ARG-DesignerBase model=Deepseek-V3-671B-Instruct, Var. NS=true, Flex. State=false2026.05 | 87.96 | |
| AgentPruneBase model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 87.92 | |
| SC (CoT)Base model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 87.85 | |
| G-DesignerBase model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 87.61 | |
| MASround=TBase model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 87.5 | |
| CoT-SCBackbone=deepseek-v3, n=52026.04 | 87.4 | |
| Hierarchical MASPrompt Opt.=false, Joint Opt.=false2026.05 | 87.01 | |
| Hierarchical MAS + MASPOPrompt Opt.=true, Joint Opt.=true2026.05 | 87.01 | |
| CLoTBackbone=gpt-42026.04 | 86.9 | |
| AutoGenBase model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 86.71 | |
| MASround=1Base model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 86.67 | |
| Hierarchical MAS + SPOPrompt Opt.=true, Joint Opt.=false2026.05 | 86.61 | |
| ARBackbone=deepseek-v32026.04 | 86.6 | |
| DMoABackbone=gpt-oss-120b, Evaluation Paradigm=few-shot2026.05 | 86.6 | |
| GoAgent2026.03 | 86.45 | |
| Hierarchical MAS + TPEPrompt Opt.=true, Joint Opt.=false2026.05 | 86.45 | |
| AgentDropoutPrompt Opt.=false, Joint Opt.=false2026.05 | 86.23 | |
| AgentVerseBase model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 85.9 | |
| CLoTBackbone=gpt-4o-mini2026.04 | 85.8 | |
| Sequential MAS + MASPOPrompt Opt.=true, Joint Opt.=true2026.05 | 85.56 | |
| ARG-Designer2026.03 | 85.51 | |
| CoTBase model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 85.42 | |
| Sequential MAS + TPEPrompt Opt.=true, Joint Opt.=false2026.05 | 84.92 | |
| VanillaBase model=Deepseek-V3-671B-Instruct, Var. NS=false, Flex. State=false2026.05 | 84.58 | |
| ISP-CoTBackbone=gpt-42026.04 | 84.2 | |
| CoTBackbone=deepseek-v32026.04 | 84.2 | |
| ISP-CoTBackbone=gpt-4o-mini2026.04 | 83.7 | |
| Sequential MASPrompt Opt.=false, Joint Opt.=false2026.05 | 83.47 | |
| EIB-LEARNER2026.03 | 83.46 | |
| ATARBase Model=Qwen3-30B2025.10 | 83.46 | |
| COPROModel=GPT-4o2026.05 | 83.42 | |
| GEPALLM=GPT-4o2026.05 | 83.42 | |
| COPROLLM=GPT-4o2026.05 | 83.42 | |
| C-CoTBackbone=gpt-4o-mini2026.04 | 83.3 | |
| PromptAgentLLM=GPT-4o2026.05 | 83.29 | |
| CoT-SCBackbone=gpt-4o-mini, n=52026.04 | 83.1 | |
| CoT-SCBackbone=gpt-4, n=52026.04 | 83.1 | |
| RADARAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 82.84 | |
| CoTBackbone=gpt-42026.04 | 82.7 | |
| C-CoTBackbone=gpt-42026.04 | 82.7 | |
| STEERBackbone=gpt-oss-120b2026.05 | 82.66 | |
| BaselineLLM=GPT-4o2026.05 | 82.63 | |
| Self-RefinePrompt Opt.=false, Joint Opt.=false2026.05 | 82.28 | |
| SC (COT×5)Autonomous Topology=false, Base LLM=gpt-4o-mini, Number of Agents=12026.05 | 82.28 | |
| SC (CoT)Prompt Opt.=false, Joint Opt.=false2026.05 | 82.17 | |
| CoTPrompt Opt.=false, Joint Opt.=false2026.05 | 81.89 | |
| MIPROModel=GPT-4o2026.05 | 81.84 | |
| MIPROLLM=GPT-4o2026.05 | 81.84 | |
| G-Designer2026.03 | 81.6 | |
| AgentSquareAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 81.5 | |
| CoTBackbone=gpt-4o-mini2026.04 | 81.4 | |
| SpecReasonBackbone=gpt-oss-120b2026.05 | 81.36 | |
| ProTeGiLLM=GPT-4o2026.05 | 81.32 | |
| Sequential MAS + SPOPrompt Opt.=true, Joint Opt.=false2026.05 | 81.1 | |
| ARG-DesignerAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 81.1 | |
| AgentDropout2026.03 | 80.94 | |
| AgentPruneAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 80.93 | |
| DALAAgent Category=Multi-Agent2025.11 | 80.92 | |
| AFlowAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 80.9 | |
| G-DesignerAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 80.75 | |
| GPTSwarmAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 80.71 | |
| AgentPrune2026.03 | 80.51 | |
| SafeSieveBackbone=gpt-oss-120b2026.05 | 80.4 | |
| MaASAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 80.25 | |
| ATARBase Model=Qwen3-4B2025.10 | 79.92 | |
| TrBackbone=gpt-42026.04 | 79.9 | |
| SWIFTEvaluation Protocol=Out-of-distribution2026.04 | 79.9 | |
| LLM-DebateAutonomous Topology=false, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 79.7 | |
| DyLANAutonomous Topology=false, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 79.59 | |
| VanillaPrompt Opt.=false, Joint Opt.=false2026.05 | 79.53 | |
| ARBackbone=gpt-42026.04 | 79.5 | |
| ARG-DesignerBackbone=gpt-oss-120b, Evaluation Paradigm=few-shot2026.05 | 79.48 | |
| GTDAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 79.44 | |
| MultiPersonaAutonomous Topology=false, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 79.23 | |
| MacNetAutonomous Topology=false, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 79.23 | |
| ATARBase Model=Phi-42025.10 | 79.13 | |
| LLM-BlenderAutonomous Topology=false, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 78.99 | |
| VanillaAutonomous Topology=false, Base LLM=gpt-4o-mini, Number of Agents=12026.05 | 78.92 | |
| AutoAgentsAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 78.65 | |
| AgentPrune-RAgent Category=Multi-Agent2025.11 | 78.6 | |
| AFlowBackbone=gpt-oss-120b2026.05 | 78.2 | |
| ADASAutonomous Topology=true, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 77.71 | |
| LLM-Debate2026.03 | 77.65 | |
| ComplexCoTAgent Category=Single-Agent2025.11 | 77.58 | |
| LLM-DebateAgent Category=Multi-Agent2025.11 | 77.52 | |
| ARBackbone=gpt-4o-mini2026.04 | 77.5 | |
| AgentVerseAutonomous Topology=false, Base LLM=gpt-4o-mini, Number of Agents=52026.05 | 77.47 |