Code Generation on MBPP (Score(%), Cost($))
87.53Execution ScoreSC-MAS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SC-MASLLM=llm pool2026.01 | 87.53 | 0.913 | |
| MasRouterLLM=llm pool2026.01 | 84 | 1.039 | |
| AFlowLLM=gpt-4o-mini2026.01 | 82.2 | 1.723 | |
| AFlowLLM=gemini-1.5-flash2026.01 | 76 | 1.832 | |
| AgentPruneLLM=gemini-1.5-flash2026.01 | 75.6 | 1.352 | |
| Complete GraphLLM=gpt-4o-mini2026.01 | 75.2 | 3.088 | |
| RouterDCLLM=llm pool2026.01 | 75.2 | 0.145 | |
| AgentPruneLLM=gpt-4o-mini2026.01 | 75 | 1.215 | |
| FrugalGPTLLM=llm pool2026.01 | 74.4 | 0.139 | |
| Complete GraphLLM=gemini-1.5-flash2026.01 | 74.2 | 1.215 | |
| LLM DebateLLM=gpt-4o-mini2026.01 | 73.6 | 4.427 | |
| IOLLM=claude-3.5-haiku2026.01 | 73.4 | 0.146 | |
| LLM DebateLLM=gemini-1.5-flash2026.01 | 73.4 | 4.529 | |
| IOLLM=gemini-1.5-flash2026.01 | 73 | 0.157 | |
| IOLLM=gpt-4o-mini2026.01 | 72.2 | 0.143 | |
| IOLLM=llama-3.1-70b2026.01 | 68.2 | 0.105 |