Code Generation on MBPP (pass@1, Time)
74.7Pass@1GraphFlow
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GraphFlowModel=Qwen-2.5-7B2026.05 | 74.7 | 16.49 | |
| GraphFlowModel=Gemma-2-9B2026.05 | 72.8 | 15.45 | |
| GraphFlowModel=Llama-3.1-8B2026.05 | 72.6 | 15.15 | |
| AFlowModel=Qwen-2.5-7B2026.05 | 68.4 | 18.8 | |
| AgentKBModel=Qwen-2.5-7B2026.05 | 67.9 | 26.07 | |
| AutoFlowModel=Qwen-2.5-7B2026.05 | 67.2 | 22.15 | |
| MetaGPTModel=Qwen-2.5-7B2026.05 | 66.7 | 13.21 | |
| TaskWeaverModel=Qwen-2.5-7B2026.05 | 66.1 | 24.67 | |
| AFlowModel=Gemma-2-9B2026.05 | 66.1 | 17.15 | |
| AFlowModel=Llama-3.1-8B2026.05 | 65.9 | 17.45 | |
| LLMCompilerModel=Qwen-2.5-7B2026.05 | 65.5 | 14.58 | |
| AgentKBModel=Gemma-2-9B2026.05 | 65.5 | 15.05 | |
| AutoFlowModel=Gemma-2-9B2026.05 | 65.1 | 18.2 | |
| LLMCompilerModel=Gemma-2-9B2026.05 | 64.8 | 14.55 | |
| AutoFlowModel=Llama-3.1-8B2026.05 | 64.5 | 18.9 | |
| LLMCompilerModel=Llama-3.1-8B2026.05 | 64.1 | 18.15 | |
| AgentKBModel=Llama-3.1-8B2026.05 | 63.8 | 18.6 | |
| TaskWeaverModel=Llama-3.1-8B2026.05 | 63.6 | 28.35 | |
| MetaGPTModel=Gemma-2-9B2026.05 | 63.4 | 13.61 | |
| TaskWeaverModel=Gemma-2-9B2026.05 | 63.2 | 24.75 | |
| MetaGPTModel=Llama-3.1-8B2026.05 | 62.7 | 17.38 | |
| VanillaModel=Qwen-2.5-7B2026.05 | 62.5 | 7.09 | |
| VanillaModel=Gemma-2-9B2026.05 | 60.5 | 9.78 | |
| VanillaModel=Llama-3.1-8B2026.05 | 59.6 | 10.25 |