Mathematical Reasoning on AMC (Perf., Cost)
97.6AccuracyQwen3-235B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3-235BMethod Category=Heuristic, Parameters=235B2026.06 | 97.6 | 0.8389 | |
| ENTROROUTERMethod Category=Routing Architecture2026.06 | 96.6 | 0.3658 | |
| FORCMethod Category=Existing Routing Architecture2026.06 | 96.2 | 0.5295 | |
| RouteLLMMethod Category=Existing Routing Architecture2026.06 | 95.8 | 0.0848 | |
| Qwen3-30BMethod Category=Heuristic, Parameters=30B2026.06 | 95.6 | 0.3893 | |
| RandomMethod Category=Heuristic2026.06 | 95.3 | 0.4213 | |
| Qwen3-4BMethod Category=Heuristic, Parameters=4B2026.06 | 93.8 | 0.031 | |
| Router-R1Method Category=Existing Routing Architecture2026.06 | 72.7 | 0.0241 | |
| xRouterMethod Category=Existing Routing Architecture2026.06 | 72.3 | 0.01 | |
| FlowBankExecutor LLM=GPT-4o mini, Optimizer LLM=Qwen3-8B2026.06 | 67.94 | 2.49 | |
| AFlow (GPT-4o)Executor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 63.77 | 4.86 | |
| MaASExecutor LLM=GPT-4o mini2026.06 | 63.36 | 2.7 | |
| AFlow (Qwen3-8B)Executor LLM=GPT-4o mini, Optimizer LLM=Qwen3-8B2026.06 | 62.75 | 2.51 | |
| ScoreFlowExecutor LLM=GPT-4o mini, Generator LLM=Qwen3-8B2026.06 | 60.15 | 3.51 | |
| AgentSquareExecutor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 58.78 | 1.79 | |
| GPTSwarmExecutor LLM=GPT-4o mini2026.06 | 55.42 | 9.04 | |
| MedPromptExecutor LLM=GPT-4o mini2026.06 | 55.19 | 5.49 | |
| Self-ConsistencyExecutor LLM=GPT-4o mini2026.06 | 53.89 | 3.44 | |
| Multi-agent DebateExecutor LLM=GPT-4o mini2026.06 | 53.82 | 7.96 | |
| MultiPersonaExecutor LLM=GPT-4o mini2026.06 | 53.74 | 0.58 | |
| IOExecutor LLM=GPT-4o mini2026.06 | 53.59 | 0.53 | |
| MENTORBackbone=Qwen2.5-7B-Base2025.10 | 53.1 | — | |
| Self-RefineExecutor LLM=GPT-4o mini2026.06 | 52.88 | 1.06 | |
| CoTExecutor LLM=GPT-4o mini2026.06 | 52.47 | 0.54 | |
| ComplexCoTExecutor LLM=GPT-4o mini2026.06 | 49.52 | 0.64 | |
| QuestABackbone=Qwen2.5-7B-Base2025.10 | 47.4 | — | |
| ADASExecutor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 47.33 | 3.22 | |
| LUFFYBackbone=Qwen2.5-7B-Base2025.10 | 46.4 | — | |
| On-policy RLBackbone=Qwen2.5-7B-Base2025.10 | 46 | — | |
| MENTORBackbone=Qwen2.5-3B-Base2025.10 | 34.2 | — | |
| QuestABackbone=Qwen2.5-3B-Base2025.10 | 34.1 | — | |
| LUFFYBackbone=Qwen2.5-3B-Base2025.10 | 32.8 | — | |
| On-policy RLBackbone=Qwen2.5-3B-Base2025.10 | 32.2 | — | |
| BaseBackbone=Qwen2.5-7B-Base2025.10 | 26.5 | — | |
| BaseBackbone=Qwen2.5-3B-Base2025.10 | 17.7 | — | |
| MENTORBackbone=LLaMa3.1-8B-Base2025.10 | 10.4 | — | |
| LUFFYBackbone=LLaMa3.1-8B-Base2025.10 | 8.4 | — | |
| On-policy RLBackbone=LLaMa3.1-8B-Base2025.10 | 8 | — | |
| QuestABackbone=LLaMa3.1-8B-Base2025.10 | 5.3 | — | |
| BaseBackbone=LLaMa3.1-8B-Base2025.10 | 1.8 | — |