Efficiency-aware code generation on EffiBench-X Python
31.1Execution TimeDirect
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DirectBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 31.1 | 49.13 | 46.28 | |
| Self-ReflectionBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 48.49 | 49.82 | 50.89 | |
| SE-AgentBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 48.57 | 50.39 | 49.63 | |
| AlphaEvolveBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 48.7 | 50.33 | 51.47 | |
| CSEBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 49.06 | 50.99 | 52.38 | |
| DirectBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 52.9 | 58.89 | 56.22 | |
| Self-ReflectionBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 57.35 | 60.2 | 61.18 | |
| AlphaEvolveBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 58.86 | 60.04 | 60.24 | |
| SE-AgentBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 60.09 | 59.68 | 60.69 | |
| DirectBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 60.21 | 62.31 | 61.66 | |
| CSEBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 61.06 | 61.03 | 62.8 | |
| SE-AgentBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 61.74 | 64.51 | 65.36 | |
| Self-ReflectionBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 62.21 | 63.09 | 63.72 | |
| AlphaEvolveBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 63.3 | 63.96 | 65.53 | |
| SE-AgentBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 64.14 | 70.58 | 67.25 | |
| DirectBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 65.43 | 69.03 | 67.31 | |
| CSEBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 65.46 | 66.82 | 68.1 | |
| AlphaEvolveBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 69.45 | 69.5 | 70.57 | |
| Self-ReflectionBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 69.79 | 71.16 | 73.08 | |
| CSEBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 71.17 | 70.39 | 73.94 |