Mathematical Reasoning on MATH (Accuracy and Token Usage)
96.67AccuracyFull-Graph
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Full-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 96.67 | — | — | |
| HieraMASMulti=true, Topo=true, Role=true, Node=true, Backbone=GPT-5-Mini2026.02 | 96.67 | — | — | |
| Full-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 95.56 | — | — | |
| AFlowMulti=true, Topo=true, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 95.56 | — | — | |
| HieraMASMulti=true, Topo=true, Role=true, Node=true, Backbone=Qwen3-80B2026.02 | 95.56 | — | — | |
| Self-Consistency+CoTMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 94.44 | — | — | |
| LLM-DebateMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 94.44 | — | — | |
| LLM-DebateMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 94.44 | — | — | |
| Random-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 94.44 | — | — | |
| 4B-InstructModel=4B-Instruct, RL Algorithm=HACPO, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 94.3 | — | — | |
| 4B-InstructModel=4B-Instruct, RL Algorithm=GSPO, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 94 | — | — | |
| 4B-InstructModel=4B-Instruct, RL Algorithm=GSPO×2, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 93.9 | — | — | |
| 4B-InstructModel=4B-Instruct, RL Algorithm=Base, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 93.7 | — | — | |
| Self-ConsistencyMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 93.33 | — | — | |
| Random-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 93.33 | — | — | |
| 4B-InstructModel=4B-Instruct, RL Algorithm=GRPO, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 93.3 | — | — | |
| CoTMulti=false, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 92.22 | — | — | |
| Self-Consistency+CoTMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 92.22 | — | — | |
| Self-ConsistencyMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 91.11 | — | — | |
| GDesignerMulti=true, Topo=true, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 91.11 | — | — | |
| MASRouterMulti=true, Topo=true, Role=true, Node=false, Backbone=GPT-5-Mini2026.02 | 91.11 | — | — | |
| CODABackbone=Qwen3-14B, Sampling Strategy=mean@82026.03 | 91.1 | 1,605 | — | |
| 4BModel=4B, RL Algorithm=HACPO, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 90.5 | — | — | |
| VLPBackbone=Qwen3-14B, Sampling Strategy=mean@82026.03 | 90.4 | 1,821 | — | |
| GRPOBackbone=Qwen3-14B, Sampling Strategy=mean@82026.03 | 90.3 | 2,229 | — | |
| CoTMulti=false, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 90 | — | — | |
| ASRRBackbone=Qwen3-14B, Sampling Strategy=mean@82026.03 | 89.1 | 1,324 | — | |
| 4BModel=4B, RL Algorithm=GRPO, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 88.9 | — | — | |
| MASRouterMulti=true, Topo=true, Role=true, Node=false, Backbone=Qwen3-80B2026.02 | 88.88 | — | — | |
| GRPOBackbone=Qwen3-8B, Sampling Strategy=mean@82026.03 | 87.8 | 2,270 | — | |
| GDesignerMulti=true, Topo=true, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 87.77 | — | — | |
| CODABackbone=Qwen3-8B, Sampling Strategy=mean@82026.03 | 87.6 | 1,700 | — | |
| 4BModel=4B, RL Algorithm=GSPO×2, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 87.5 | — | — | |
| VLPBackbone=Qwen3-8B, Sampling Strategy=mean@82026.03 | 87.5 | 1,801 | — | |
| 4BModel=4B, RL Algorithm=GSPO, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 87 | — | — | |
| ASRRBackbone=Qwen3-8B, Sampling Strategy=mean@82026.03 | 86.7 | 1,598 | — | |
| GRPOBackbone=Qwen3-4B, Sampling Strategy=mean@82026.03 | 85.3 | 2,774 | — | |
| CODABackbone=Qwen3-4B, Sampling Strategy=mean@82026.03 | 85.1 | 1,914 | — | |
| ASRRBackbone=Qwen3-4B, Sampling Strategy=mean@82026.03 | 84.8 | 1,464 | — | |
| 4B-InstructModel=4B-Instruct, RL Algorithm=Naive, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 84.5 | — | — | |
| VLPBackbone=Qwen3-4B, Sampling Strategy=mean@82026.03 | 84.5 | 1,716 | — | |
| AFlowMulti=true, Topo=true, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 84.44 | — | — | |
| AbstRaLModel=Qwen2.5-Math-7B-Instruct, GSM Train Method=AbstRaL, Evaluation Protocol=Zero-shot2025.06 | 83.9 | — | — | |
| CoAModel=Qwen2.5-Math-7B-Instruct, GSM Train Method=CoA, Evaluation Protocol=Zero-shot2025.06 | 83.7 | — | — | |
| 4BModel=4B, RL Algorithm=Base, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 83.6 | — | — | |
| Ori-SFTModel=Qwen2.5-Math-7B-Instruct, GSM Train Method=Ori-SFT, Evaluation Protocol=Zero-shot2025.06 | 83.5 | — | — | |
| CoT-RLModel=Qwen2.5-Math-7B-Instruct, GSM Train Method=CoT-RL, Evaluation Protocol=Zero-shot2025.06 | 82.9 | — | — | |
| HACPOModel Backbone=Qwen3-8B-Base2026.03 | 81.3 | — | — | |
| ROSA2Model=Qwen3-8B2026.03 | 80.8 | — | — | |
| Zero-shot-EIModel=Gemini-1.5-Pro, Model Role=Teacher Model2026.02 | 80.59 | — | — | |
| HACPOModel Backbone=Qwen3-4B-Base2026.03 | 80.3 | — | — | |
| 4B-BaseModel=4B-Base, RL Algorithm=HACPO, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 80.1 | — | — | |
| 4B-BaseModel=4B-Base, RL Algorithm=GSPO×2, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 79.4 | — | — | |
| qwen3-4BModel=qwen3-4B, RL Algorithm=GSPO×2, Heterogeneity Setting=Qwen3-4B-Base + Llama3.2-3B-Instruct2026.03 | 79.4 | — | — | |
| GRPOModel Backbone=Qwen3-4B-Base2026.03 | 78.8 | — | — | |
| 4B-BaseModel=4B-Base, RL Algorithm=GRPO, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 78.8 | — | — | |
| qwen3-4BModel=qwen3-4B, RL Algorithm=GRPO, Heterogeneity Setting=Qwen3-4B-Base + Llama3.2-3B-Instruct2026.03 | 78.8 | — | — | |
| 4B-BaseModel=4B-Base, RL Algorithm=GSPO, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 78.7 | — | — | |
| qwen3-4BModel=qwen3-4B, RL Algorithm=GSPO, Heterogeneity Setting=Qwen3-4B-Base + Llama3.2-3B-Instruct2026.03 | 78.7 | — | — | |
| qwen3-4BModel=qwen3-4B, RL Algorithm=HACPO, Heterogeneity Setting=Qwen3-4B-Base + Llama3.2-3B-Instruct2026.03 | 78.3 | — | — | |
| BaseMulti=false, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 77.78 | — | — | |
| Zero-shotModel=Gemini-1.5-Pro, Model Role=Teacher Model2026.02 | 77.2 | — | — | |
| ExGRPOModel=Qwen2.5-7B-Instruct, Model Role=Student Model, Adapter=On-Policy Adapt.2026.02 | 77.12 | — | — | |
| ExGRPOModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 76.58 | — | — | |
| RevThinkModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 76.4 | — | — | |
| On-Policy DistillationModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 76.24 | — | — | |
| Answer AugModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 76.18 | — | — | |
| Divide-or-ConquerModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 76.05 | — | — | |
| Question AugModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 75.97 | — | — | |
| Rephrase QuestionModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 75.64 | — | — | |
| Distill Step-by-StepModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 75.32 | — | — | |
| SKDModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 74.88 | — | — | |
| BaseMulti=false, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 74.44 | — | — | |
| Zero-shotModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 74.4 | — | — | |
| Llama 3 405Bshot=0-shot, CoT=true2024.07 | 73.8 | — | — | |
| 4BModel=4B, RL Algorithm=Naive, Heterogeneity Setting=Qwen3-4B + Qwen3-4B-Instruct2026.03 | 73.7 | — | — | |
| M2CLBackbone=Qwen-72B, Number of LLMs=82026.02 | 72.5 | — | — | |
| Qwen3-14B-BaseBackbone=Qwen3-14B, Sampling Strategy=mean@82026.03 | 72.5 | 914 | — | |
| 4B-BaseModel=4B-Base, RL Algorithm=Naive, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 71.2 | — | — | |
| qwen3-4BModel=qwen3-4B, RL Algorithm=Naive, Heterogeneity Setting=Qwen3-4B-Base + Llama3.2-3B-Instruct2026.03 | 71.2 | — | — | |
| Claude 3.5 Sonnetshot=0-shot, CoT=false2024.07 | 71.1 | — | — | |
| ROSA2Model=Qwen3-0.6B-Instruct2026.03 | 70.8 | — | — | |
| ROSA2Model=Qwen2.5-7B-Base2026.03 | 68.4 | — | — | |
| BaseModel Backbone=Qwen3-4B-Base2026.03 | 67.6 | — | — | |
| 4B-BaseModel=4B-Base, RL Algorithm=Base, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 67.6 | — | — | |
| qwen3-4BModel=qwen3-4B, RL Algorithm=Base, Heterogeneity Setting=Qwen3-4B-Base + Llama3.2-3B-Instruct2026.03 | 67.6 | — | — | |
| ROSAModel=Qwen3-8B, Update Location=LM Head, Reward Model=Model-based2025.09 | 67.4 | — | — | |
| 1.7B-BaseModel=1.7B-Base, RL Algorithm=HACPO, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 67.4 | — | — | |
| Qwen3-8B-BaseBackbone=Qwen3-8B, Sampling Strategy=mean@82026.03 | 67.3 | 851 | — | |
| ROSAModel=Qwen3-0.6B-Instruct2026.03 | 66.2 | — | — | |
| Llama 3.1 InstructModel Scale=70B2025.04 | 66.2 | — | — | |
| GPT-42024.03 | 66.1 | — | — | |
| HACPOModel Backbone=Qwen3-1.7B-Base2026.03 | 66.1 | — | — | |
| ROSAModel=Qwen3-8B, Update Location=LM Head, Reward Model=Rule-based2025.09 | 65.8 | — | — | |
| ROSAModel=Qwen3-8B, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 65.8 | — | — | |
| ROSAModel=Qwen2.5-7B-Instruct, Update Location=LM Head, Reward Model=Model-based2025.09 | 65.2 | — | — | |
| 1.7B-BaseModel=1.7B-Base, RL Algorithm=GRPO, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 65.2 | — | — | |
| TextGradModel=Qwen3-0.6B-Instruct2026.03 | 65 | — | — | |
| 1.7B-BaseModel=1.7B-Base, RL Algorithm=GSPO×2, Heterogeneity Setting=Qwen3-1.7B-Base + Qwen3-4B-Base2026.03 | 65 | — | — | |
| ℓ-MoEaccActive Params=8.44B, Total Params=94.8B2026.01 | 64.88 | — | — |