Code Generation on DS-1000 (Pass@1)
58.65Pass@1ST-EVO
Evaluation results
| Method | Links | |
|---|---|---|
| ST-EVOBackbone=gpt-oss-120b2026.02 | 58.65 | |
| FlowSteerMethodology=Ours, Backbone=GPT-4o-mini2026.02 | 58.59 | |
| ARG-DesignerBackbone=gpt-oss-120b2026.02 | 55.6 | |
| G-DesignerBackbone=gpt-oss-120b2026.02 | 54.82 | |
| AFlow (4o-mini)Methodology=AFlow, Backbone=GPT-4o-mini2026.02 | 53.91 | |
| SafeSieveBackbone=gpt-oss-120b2026.02 | 53.73 | |
| GPTSwarmBackbone=gpt-oss-120b2026.02 | 52.35 | |
| Orchestrator (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Orchestrator2026.02 | 51.56 | |
| STEERBackbone=gpt-oss-120b2026.02 | 51.34 | |
| SpecReasonBackbone=gpt-oss-120b2026.02 | 50.08 | |
| LLM-DebateBackbone=gpt-oss-120b2026.02 | 49.26 | |
| AutoGenBackbone=gpt-oss-120b2026.02 | 48.55 | |
| LangGraphBackbone=gpt-oss-120b2026.02 | 47.95 | |
| Random GraphBackbone=gpt-oss-120b2026.02 | 47.48 | |
| Agentflow (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Agentflow2026.02 | 46.88 | |
| AFlowBackbone=gpt-oss-120b2026.02 | 46.82 | |
| Baseline (4o-mini)Methodology=Baseline, Backbone=GPT-4o-mini2026.02 | 45.31 | |
| Complete GraphBackbone=gpt-oss-120b2026.02 | 45.05 | |
| TreeBackbone=gpt-oss-120b2026.02 | 43.5 | |
| Router-R1 (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Router-R12026.02 | 42.97 | |
| CoTBackbone=gpt-oss-120b2026.02 | 42.35 | |
| StarBackbone=gpt-oss-120b2026.02 | 41.82 | |
| VanillaBackbone=gpt-oss-120b2026.02 | 38.4 | |
| GRPO (Qwen3-8B)Methodology=GRPO, Backbone=Qwen3-8B2026.02 | 38.28 | |
| ChainBackbone=gpt-oss-120b2026.02 | 38.25 | |
| SCBackbone=gpt-oss-120b2026.02 | 37.66 | |
| Baseline (Qwen3-8B)Methodology=Baseline, Backbone=Qwen3-8B2026.02 | 34.38 | |
| SFT (Qwen3-8B)Methodology=SFT, Backbone=Qwen3-8B2026.02 | 25.78 |