List your company
ResearchFormerly WizWand

Code Generation on HumanEval (Accuracy, Latency)

90.08AccuracyVanilla MAS
50.965661.120371.27581.4297May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Random, Training-Free (TF)=true2026.0590.0859.27
AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.0588.9838.26
Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Star, Training-Free (TF)=true2026.0588.7159.79
Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Layered, Training-Free (TF)=true2026.0588.7153.2
SC-CoTBackbone=Qwen2.5-14B-Instruct, Topology=-, Training-Free (TF)=true2026.0588.4316.49
Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.0588.1559.12
Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Chain, Training-Free (TF)=true2026.0587.8860.95
SC-CoTBackbone=Qwen2.5-72B-Instruct, Topology=-, Training-Free (TF)=true2026.0587.0558.82
AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Random, Training-Free (TF)=false2026.0587.0549.9
Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Layered, Training-Free (TF)=true2026.0586.516.27
AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Layered, Training-Free (TF)=false2026.0586.549.34
CONCATBackbone=Qwen2.5-14B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.0586.1913.55
LLM-DebateBackbone=Qwen2.5-14B-Instruct, Topology=Debate, Training-Free (TF)=true2026.0586.1818.96
Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Chain, Training-Free (TF)=true2026.0585.9520.17
LLM-DebateBackbone=Qwen2.5-72B-Instruct, Topology=Debate, Training-Free (TF)=true2026.0585.9557.46
AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Star, Training-Free (TF)=false2026.0585.9552.73
AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Star, Training-Free (TF)=false2026.0585.6718.44
AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Layered, Training-Free (TF)=false2026.0585.6720.33
Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Star, Training-Free (TF)=true2026.0585.1222.06
AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.0584.8512.78
Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Random, Training-Free (TF)=true2026.0584.5720.32
Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.0584.5720.21
AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Chain, Training-Free (TF)=false2026.0584.5771.63
CoTBackbone=Qwen2.5-72B-Instruct, Topology=-, Training-Free (TF)=true2026.0584.310.99
CONCATBackbone=Qwen2.5-72B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.0584.346.31
CoTBackbone=Qwen2.5-14B-Instruct, Topology=-, Training-Free (TF)=true2026.0583.23.06
AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Chain, Training-Free (TF)=false2026.0581.8224.45
AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Random, Training-Free (TF)=false2026.0581.4418.41
CoTBackbone=Llama3-8B-Instruct, Topology=-, Training-Free (TF)=true2026.0563.913.28
SC-CoTBackbone=Llama3-8B-Instruct, Topology=-, Training-Free (TF)=true2026.0563.3616.89
CONCATBackbone=Llama3-8B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.0562.2623.77
AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Chain, Training-Free (TF)=false2026.0561.93104.8
LLM-DebateBackbone=Llama3-8B-Instruct, Topology=Debate, Training-Free (TF)=true2026.0560.8884.11
Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Chain, Training-Free (TF)=true2026.0558.1365.29
AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Star, Training-Free (TF)=false2026.0557.356.97
AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Random, Training-Free (TF)=false2026.0557.0261.14
Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Star, Training-Free (TF)=true2026.0556.91103.28
Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Layered, Training-Free (TF)=true2026.0556.7579.27
AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.0556.75105.81
Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Random, Training-Free (TF)=true2026.0556.280.85
Vanilla MASBackbone=Llama3-8B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.0553.9990.6
AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Layered, Training-Free (TF)=false2026.0552.47113.07