Code Generation on HumanEval (Score and Cost)
92.37Score (%)SC-MAS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SC-MASLLM=llm pool2026.01 | 92.37 | 0.154 | |
| MasRouterLLM=llm pool2026.01 | 90.52 | 0.185 | |
| AFlowLLM=gpt-4o-mini2026.01 | 90.15 | 0.363 | |
| RouterDCLLM=llm pool2026.01 | 87.75 | 0.023 | |
| FrugalGPTLLM=llm pool2026.01 | 87.31 | 0.026 | |
| AgentPruneLLM=gpt-4o-mini2026.01 | 86.8 | 0.254 | |
| IOLLM=claude-3.5-haiku2026.01 | 86.33 | 0.025 | |
| IOLLM=gpt-4o-mini2026.01 | 85.71 | 0.025 | |
| AFlowLLM=gemini-1.5-flash2026.01 | 85.69 | 0.386 | |
| Complete GraphLLM=gpt-4o-mini2026.01 | 85 | 0.488 | |
| LLM DebateLLM=gpt-4o-mini2026.01 | 84.38 | 0.624 | |
| Complete GraphLLM=gemini-1.5-flash2026.01 | 83.75 | 0.568 | |
| IOLLM=gemini-1.5-flash2026.01 | 82.61 | 0.032 | |
| AgentPruneLLM=gemini-1.5-flash2026.01 | 82.55 | 0.271 | |
| IOLLM=llama-3.1-70b2026.01 | 80.75 | 0.013 | |
| LLM DebateLLM=gemini-1.5-flash2026.01 | 79.38 | 0.693 |