List your company
ResearchFormerly WizWand

Code Generation on MBPP (Acc (%), Avg Cost)

49.23Accuracy (%)MACA (Ours)
11.030820.947930.86540.7821May 25, 2026
Updated 5mo ago

Evaluation results

MethodLinks
MACA (Ours)Model=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✓2026.0549.232,412.3
AgentVerseModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0548.945,017.5
MacNetModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0547.836,794.8
Self-RefineModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0547.53498.7
PuppeteerModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✓2026.0547.173,696.5
SC (CoT×5)Model=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0546.361,206.9
VanillaModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0545.97138.4
CoTModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0544.62146.1
ComplexCoTModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0544.18247.4
MaASModel=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✗2026.0542.053,215.7
AgentPruneModel=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✗2026.0531.442,787.8
DyLANModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0512.511,647.8