Code Generation on APPS Interview
2.64Pass@1Codex
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CodexBackbone=Codex-12B, Filtering=Filtered, Shot=1-shot2021.07 | 2.64 | 5.78 | 3.23 | 7.13 | — | — | — | — | — | — | — | — | — | |
| LCPLLM backbone=GPT-4o2026.03 | 0.775 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LPWLLM backbone=GPT-4o2026.03 | 0.652 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-NeoBackbone=GPT-Neo 2.7B, Filtering=Raw, Shot=Zero-shot2021.07 | 0.57 | — | 0.8 | — | — | — | — | — | — | — | — | — | — | |
| LDBLLM backbone=GPT-4o2026.03 | 0.522 | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaselineLLM backbone=GPT-4o2026.03 | 0.435 | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodexBackbone=Codex-12B, Filtering=Raw, Shot=1-shot2021.07 | 0.14 | 0.3 | 0.51 | 1.02 | 2.04 | 7.94 | 3.7 | — | — | — | — | — | — | |
| CODETModel=code-davinci-002, Setting=zero-shot, Temperature=0.8, Sampling number=502022.07 | 0.081 | — | — | — | — | — | — | — | 0.112 | 0.181 | — | — | — | |
| BaselineModel=code-davinci-002, Setting=zero-shot, Temperature=0.8, Sampling number=502022.07 | 0.051 | — | — | — | — | — | — | — | — | 0.128 | 0.23 | — | — |