Algebraic Reasoning on AQUA
91.89AccuracySafeSieve
Evaluation Results
| Method | Links | |
|---|---|---|
| SafeSieveParadigm=post-prune, Base model=deepseek-V3-671B2025.08 | 91.89 | |
| AgentDropoutParadigm=post-prune, Base model=deepseek-V3-671B2025.08 | 91.37 | |
| AgentPruneParadigm=post-prune, Base model=deepseek-V3-671B2025.08 | 90.3 | |
| Weighted MV2026.04 | 90.23 | |
| Simple MV2026.04 | 90.16 | |
| EMS-SimSorting Strategy=Semantic Similarity2026.04 | 90.16 | |
| EMS-RelSorting Strategy=Historical Reliability2026.04 | 90.16 | |
| Conf-MV2026.04 | 89.92 | |
| G-DesignerParadigm=pre-design, Base model=deepseek-V3-671B2025.08 | 89.63 | |
| Clean ModelTarget Model=Mistral-7B, Trigger Condition=Without Trigger2025.04 | 88.7 | |
| HCP-MADBase Model=Qwen2.5-32b-instruct + Llama-3.1-70b-instruct2026.04 | 88.58 | |
| Self-Consistency2026.04 | 88.52 | |
| GPTSwarmParadigm=pre-design, Base model=deepseek-V3-671B2025.08 | 88.4 | |
| ShadowCoTTarget Model=Mistral-7B, Trigger Condition=Without Trigger2025.04 | 88.3 | |
| Single Agent2026.04 | 87.66 | |
| DarkMindTarget Model=Mistral-7B, Trigger Condition=Without Trigger2025.04 | 87.4 | |
| BadChainTarget Model=Mistral-7B, Trigger Condition=Without Trigger2025.04 | 87 | |
| MADBase Model=Llama-3.1-70b-instruct2026.04 | 86.61 | |
| MADBase Model=Qwen2.5-32b-instruct2026.04 | 85.83 | |
| SCBase Model=Llama-3.1-70b-instruct2026.04 | 85.43 | |
| CoTParadigm=single, Base model=deepseek-V3-671B2025.08 | 85.42 | |
| SCBase Model=Qwen2.5-32b-instruct2026.04 | 85.04 | |
| SafeSieveParadigm=post-prune, Base model=gpt-4o-mini (~8B)2025.08 | 84.9 | |
| AgentDropoutParadigm=post-prune, Base model=gpt-4o-mini (~8B)2025.08 | 84.87 | |
| AgentPruneParadigm=post-prune, Base model=gpt-4o-mini (~8B)2025.08 | 84.71 | |
| CoTBase Model=Llama-3.1-70b-instruct2026.04 | 84.65 | |
| DOWNBase Model=Qwen2.5-32b-instruct + Llama-3.1-70b-instruct2026.04 | 84.65 | |
| VanillaParadigm=single, Base model=deepseek-V3-671B2025.08 | 84.58 | |
| Graph-GRPO2026.03 | 84.21 | |
| Heter-MADBase Model=Qwen2.5-32b-instruct + Llama-3.1-70b-instruct2026.04 | 83.86 | |
| EIB-LEARNER2026.03 | 83.49 | |
| MARSBase Model=Qwen2.5-32b-instruct + Llama-3.1-70b-instruct2026.04 | 83.07 | |
| SRBase Model=Qwen2.5-32b-instruct2026.04 | 82.68 | |
| SRBase Model=Llama-3.1-70b-instruct2026.04 | 82.28 | |
| G-designer2026.03 | 81.6 | |
| AgentDropout2026.03 | 80.94 | |
| AgentPrune2026.03 | 80.51 | |
| CoTBase Model=Qwen2.5-32b-instruct2026.04 | 80.31 | |
| G-DesignerParadigm=pre-design, Base model=gpt-4o-mini (~8B)2025.08 | 80.07 | |
| QAP50Model=GPT-4 Turbo, Constraint (N)=502024.07 | 79.1 | |
| BaselineModel=GPT-4 Turbo2024.07 | 78.7 | |
| TADBModel=GPT-4 Turbo2024.07 | 78.7 | |
| GPTSwarmParadigm=pre-design, Base model=gpt-4o-mini (~8B)2025.08 | 78.4 | |
| QAP150Model=GPT-4 Turbo, Constraint (N)=1502024.07 | 78 | |
| LLM-Debate2026.03 | 77.65 | |
| QAP25Model=GPT-4 Turbo, Constraint (N)=252024.07 | 77.6 | |
| Random2026.03 | 76.48 | |
| QAP200Model=GPT-4 Turbo, Constraint (N)=2002024.07 | 76.4 | |
| SC (CoT)2026.03 | 75.63 | |
| QAP100Model=GPT-4 Turbo, Constraint (N)=1002024.07 | 75.6 | |
| CoTModel=GPT-4 Turbo2024.07 | 74.4 | |
| Chain2026.03 | 74.05 | |
| CoT2026.03 | 73.58 | |
| Complete2026.03 | 72.95 | |
| VanillaParadigm=single, Base model=gpt-4o-mini (~8B)2025.08 | 71.42 | |
| Tree2026.03 | 71.23 | |
| Vanilla2026.03 | 71.06 | |
| CoTParadigm=single, Base model=gpt-4o-mini (~8B)2025.08 | 65 | |
| PS+Model=GPT-4 Turbo2024.07 | 52.8 | |
| Qwen3-8BCategory=Naive LLM2026.01 | 33.45 | |
| VIST2-8BCategory=Ours2026.01 | 32.1 | |
| Qwen3-VL-8BCategory=Visual-enhanced LLM2026.01 | 29.88 | |
| Qwen3-4BCategory=Naive LLM2026.01 | 28.4 | |
| VIST2-4BCategory=Ours2026.01 | 27.95 | |
| Qwen3-VL-4BCategory=Visual-enhanced LLM2026.01 | 25.96 |