Code Generation on HumanEval++
98.44AccuracyAFlow
Evaluation Results
| Method | Links | |
|---|---|---|
| AFlowMulti=true, Topo=true, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 98.44 | |
| MASRouterMulti=true, Topo=true, Role=true, Node=false, Backbone=Qwen3-80B2026.02 | 98.44 | |
| MASRouterMulti=true, Topo=true, Role=true, Node=false, Backbone=GPT-5-Mini2026.02 | 96.88 | |
| HieraMASMulti=true, Topo=true, Role=true, Node=true, Backbone=Qwen3-80B2026.02 | 96.88 | |
| AFlowMulti=true, Topo=true, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 95.31 | |
| GDesignerMulti=true, Topo=true, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 93.75 | |
| HieraMASMulti=true, Topo=true, Role=true, Node=true, Backbone=GPT-5-Mini2026.02 | 93.75 | |
| Full-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 92.19 | |
| Random-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 92.19 | |
| Self-Consistency+CoTMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 90.62 | |
| GDesignerMulti=true, Topo=true, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 90.62 | |
| BaseMulti=false, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 89.06 | |
| Self-ConsistencyMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 89.06 | |
| Full-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 89.06 | |
| CoTMulti=false, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 87.5 | |
| Self-ConsistencyMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 87.5 | |
| LLM-DebateMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 87.5 | |
| LLM-DebateMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 87.5 | |
| CoTMulti=false, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 85.94 | |
| Self-Consistency+CoTMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 85.94 | |
| Random-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 85.94 | |
| BaseMulti=false, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 84.14 | |
| TaH+Model Size=4B*2025.11 | 72 | |
| OuroModel Size=4B*2025.11 | 70.7 | |
| TaHModel Size=4B*2025.11 | 70.1 | |
| StandardModel Size=4B*2025.11 | 69.2 | |
| SoftThinkModel Size=4B*2025.11 | 69.2 | |
| TaH+Model Size=1.7B2025.11 | 51.5 | |
| TaHModel Size=1.7B2025.11 | 50 | |
| SoftThinkModel Size=1.7B2025.11 | 43.3 | |
| OuroModel Size=1.7B2025.11 | 40.9 | |
| StandardModel Size=1.7B2025.11 | 39 | |
| TaH+Model Size=0.6B2025.11 | 22 | |
| TaHModel Size=0.6B2025.11 | 21.6 | |
| OuroModel Size=0.6B2025.11 | 18.9 | |
| StandardModel Size=0.6B2025.11 | 16.8 | |
| AlwaysThinkModel Size=1.7B2025.11 | 16.4 | |
| SoftThinkModel Size=0.6B2025.11 | 14.3 | |
| AlwaysThinkModel Size=0.6B2025.11 | 9.1 |