Code Generation on HumanEval (Accuracy, Latency)
90.08AccuracyVanilla MAS
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Random, Training-Free (TF)=true2026.05 | 90.08 | 59.27 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.05 | 88.98 | 38.26 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Star, Training-Free (TF)=true2026.05 | 88.71 | 59.79 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Layered, Training-Free (TF)=true2026.05 | 88.71 | 53.2 | |
| SC-CoTBackbone=Qwen2.5-14B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 88.43 | 16.49 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.05 | 88.15 | 59.12 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Chain, Training-Free (TF)=true2026.05 | 87.88 | 60.95 | |
| SC-CoTBackbone=Qwen2.5-72B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 87.05 | 58.82 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Random, Training-Free (TF)=false2026.05 | 87.05 | 49.9 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Layered, Training-Free (TF)=true2026.05 | 86.5 | 16.27 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Layered, Training-Free (TF)=false2026.05 | 86.5 | 49.34 | |
| CONCATBackbone=Qwen2.5-14B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.05 | 86.19 | 13.55 | |
| LLM-DebateBackbone=Qwen2.5-14B-Instruct, Topology=Debate, Training-Free (TF)=true2026.05 | 86.18 | 18.96 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Chain, Training-Free (TF)=true2026.05 | 85.95 | 20.17 | |
| LLM-DebateBackbone=Qwen2.5-72B-Instruct, Topology=Debate, Training-Free (TF)=true2026.05 | 85.95 | 57.46 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Star, Training-Free (TF)=false2026.05 | 85.95 | 52.73 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Star, Training-Free (TF)=false2026.05 | 85.67 | 18.44 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Layered, Training-Free (TF)=false2026.05 | 85.67 | 20.33 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Star, Training-Free (TF)=true2026.05 | 85.12 | 22.06 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.05 | 84.85 | 12.78 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Random, Training-Free (TF)=true2026.05 | 84.57 | 20.32 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.05 | 84.57 | 20.21 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Chain, Training-Free (TF)=false2026.05 | 84.57 | 71.63 | |
| CoTBackbone=Qwen2.5-72B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 84.3 | 10.99 | |
| CONCATBackbone=Qwen2.5-72B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.05 | 84.3 | 46.31 | |
| CoTBackbone=Qwen2.5-14B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 83.2 | 3.06 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Chain, Training-Free (TF)=false2026.05 | 81.82 | 24.45 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Random, Training-Free (TF)=false2026.05 | 81.44 | 18.41 | |
| CoTBackbone=Llama3-8B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 63.91 | 3.28 | |
| SC-CoTBackbone=Llama3-8B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 63.36 | 16.89 | |
| CONCATBackbone=Llama3-8B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.05 | 62.26 | 23.77 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Chain, Training-Free (TF)=false2026.05 | 61.93 | 104.8 | |
| LLM-DebateBackbone=Llama3-8B-Instruct, Topology=Debate, Training-Free (TF)=true2026.05 | 60.88 | 84.11 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Chain, Training-Free (TF)=true2026.05 | 58.13 | 65.29 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Star, Training-Free (TF)=false2026.05 | 57.3 | 56.97 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Random, Training-Free (TF)=false2026.05 | 57.02 | 61.14 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Star, Training-Free (TF)=true2026.05 | 56.91 | 103.28 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Layered, Training-Free (TF)=true2026.05 | 56.75 | 79.27 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.05 | 56.75 | 105.81 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Random, Training-Free (TF)=true2026.05 | 56.2 | 80.85 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.05 | 53.99 | 90.6 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Layered, Training-Free (TF)=false2026.05 | 52.47 | 113.07 |