Efficiency-aware code generation on EffiBench-X C++
17.42Execution TimeDirect
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DirectBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 17.42 | 46.55 | 19.24 | |
| DirectBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 44.21 | 45.79 | 41.01 | |
| Self-ReflectionBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 45.43 | 46.53 | 43.16 | |
| AlphaEvolveBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 46.19 | 46.41 | 44.02 | |
| Self-ReflectionBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 47.47 | 48 | 44.75 | |
| SE-AgentBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 47.74 | 46.42 | 44.54 | |
| CSEBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 48.7 | 49.03 | 47.95 | |
| AlphaEvolveBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 48.75 | 48.67 | 46.58 | |
| CSEBase Model=DeepSeek-v3-0324, Test-time budget=30 candidates2026.01 | 48.75 | 46.89 | 47.37 | |
| SE-AgentBase Model=Qwen3-235B-A22B, Test-time budget=30 candidates2026.01 | 50.71 | 48.25 | 47.19 | |
| DirectBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 60.02 | 62.6 | 56.6 | |
| Self-ReflectionBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 61.79 | 62.08 | 56.76 | |
| DirectBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 63.16 | 69.16 | 62.78 | |
| AlphaEvolveBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 64.13 | 63.05 | 60.54 | |
| SE-AgentBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 68.61 | 62.7 | 63.46 | |
| CSEBase Model=GPT-5, Test-time budget=30 candidates2026.01 | 69.94 | 64.42 | 66.83 | |
| Self-ReflectionBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 70.29 | 68.42 | 67.43 | |
| AlphaEvolveBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 71.07 | 68.45 | 67.66 | |
| SE-AgentBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 72.93 | 68.81 | 70.61 | |
| CSEBase Model=Claude-4.5-Sonnet, Test-time budget=30 candidates2026.01 | 75.29 | 69.82 | 74.88 |