Function-level synthesis on LCB hard (held-out 25% eval)
33.2Utility Difference (Δπ)SR
Evaluation Results
| Method | Links | |
|---|---|---|
| SRGenerator=gpt-5-mini2026.06 | 33.2 | |
| BoNGenerator=gpt-5-mini2026.06 | 20 | |
| bayesian_greedyGenerator=gpt-5-mini2026.06 | 7.6 | |
| fixed_pipelineGenerator=gpt-5-mini2026.06 | 7.5 | |
| tCllmGenerator=gpt-5-mini2026.06 | 6 | |
| bayesian_DPGenerator=gpt-oss-20b2026.06 | 4.6 | |
| RfxGenerator=gpt-5-mini2026.06 | 4.4 | |
| tCtestGenerator=gpt-5-mini2026.06 | 4.3 | |
| tCsynGenerator=gpt-5-mini2026.06 | 3.1 | |
| bayesian_DPGenerator=claude-sonnet-4.52026.06 | 2.5 | |
| bayesian_DPGenerator=qwen3-coder2026.06 | 2.4 | |
| bayesian_DPGenerator=claude-haiku-4.52026.06 | 2.1 | |
| bayesian_DPGenerator=gpt-5-mini2026.06 | 0.3 |