Code Generation on MBPP (Perf., Cost)
84.26Performance (%)FlowBank
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FlowBankExecutor LLM=GPT-4o mini, Optimizer LLM=Qwen3-8B2026.06 | 84.26 | 1.62 | |
| AFlow (GPT-4o)Executor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 83.77 | 0.57 | |
| ScoreFlowExecutor LLM=GPT-4o mini, Generator LLM=Qwen3-8B2026.06 | 83.63 | 1.7 | |
| AFlow (Qwen3-8B)Executor LLM=GPT-4o mini, Optimizer LLM=Qwen3-8B2026.06 | 79.37 | 1.02 | |
| MaASExecutor LLM=GPT-4o mini2026.06 | 79.08 | 2.04 | |
| Multi-agent DebateExecutor LLM=GPT-4o mini2026.06 | 74.68 | 0.86 | |
| CoTExecutor LLM=GPT-4o mini2026.06 | 74.29 | 0.08 | |
| Self-ConsistencyExecutor LLM=GPT-4o mini2026.06 | 73.98 | 0.52 | |
| IOExecutor LLM=GPT-4o mini2026.06 | 72.92 | 0.08 | |
| AgentSquareExecutor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 72.73 | 0.52 | |
| ADASExecutor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 72.34 | 0.77 | |
| MedPromptExecutor LLM=GPT-4o mini2026.06 | 71.85 | 0.61 | |
| ComplexCoTExecutor LLM=GPT-4o mini2026.06 | 71.68 | 0.1 | |
| GPTSwarmExecutor LLM=GPT-4o mini2026.06 | 71.26 | 0.25 | |
| Self-RefineExecutor LLM=GPT-4o mini2026.06 | 71.07 | 0.27 | |
| MultiPersonaExecutor LLM=GPT-4o mini2026.06 | 70.38 | 0.28 |