Code Generation on HumanEval (pass@1 and Time)
86.2pass@1GraphFlow
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GraphFlowModel=Qwen-2.5-7B2026.05 | 86.2 | 15.76 | |
| GraphFlowModel=Gemma-2-9B2026.05 | 82.5 | 16.65 | |
| AFlowModel=Qwen-2.5-7B2026.05 | 78.1 | 18.55 | |
| AutoFlowModel=Qwen-2.5-7B2026.05 | 77.4 | 18.3 | |
| AgentKBModel=Qwen-2.5-7B2026.05 | 76.8 | 20.71 | |
| GraphFlowModel=Llama-3.1-8B2026.05 | 76.6 | 13.45 | |
| AFlowModel=Gemma-2-9B2026.05 | 75.4 | 19.5 | |
| TaskWeaverModel=Qwen-2.5-7B2026.05 | 75.3 | 25.76 | |
| LLMCompilerModel=Qwen-2.5-7B2026.05 | 74 | 12.29 | |
| AgentKBModel=Gemma-2-9B2026.05 | 73.8 | 18.5 | |
| AutoFlowModel=Gemma-2-9B2026.05 | 72.5 | 20.4 | |
| AFlowModel=Llama-3.1-8B2026.05 | 72.2 | 17.1 | |
| MetaGPTModel=Qwen-2.5-7B2026.05 | 72.1 | 12.2 | |
| AgentKBModel=Llama-3.1-8B2026.05 | 71.2 | 18.05 | |
| LLMCompilerModel=Gemma-2-9B2026.05 | 71.2 | 16.85 | |
| AutoFlowModel=Llama-3.1-8B2026.05 | 71 | 19.55 | |
| MetaGPTModel=Gemma-2-9B2026.05 | 70.7 | 15.69 | |
| LLMCompilerModel=Llama-3.1-8B2026.05 | 70.5 | 14.52 | |
| TaskWeaverModel=Gemma-2-9B2026.05 | 70.5 | 25.25 | |
| MetaGPTModel=Llama-3.1-8B2026.05 | 70.2 | 13.62 | |
| TaskWeaverModel=Llama-3.1-8B2026.05 | 69.9 | 24.8 | |
| VanillaModel=Qwen-2.5-7B2026.05 | 69.2 | 7.23 | |
| VanillaModel=Gemma-2-9B2026.05 | 66.8 | 13.78 | |
| VanillaModel=Llama-3.1-8B2026.05 | 66.4 | 8.76 |