Code Generation on APPS interview-level (test)
0.5717Mean Scoreno_budget (oracle)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| no_budget (oracle)alpha=0.52026.05 | 0.5717 | — | — | — | 100 | |
| no_budget (oracle)alpha=0.82026.05 | 0.5717 | — | — | — | 100 | |
| zebra-additivealpha=0.82026.05 | 0.5605 | 0.5972 | 56.2 | 0.0192 | 98.1 | |
| zebra-mult_offsetalpha=0.82026.05 | 0.5576 | 0.5957 | 55.2 | 0.0167 | 97.5 | |
| zebra-mult_offsetalpha=0.52026.05 | 0.5397 | 0.5362 | 54.1 | 0.011 | 94.4 | |
| llmalpha=0.82026.05 | 0.5389 | 0.5323 | 54 | 0.014 | 94.3 | |
| zebra-additivealpha=0.52026.05 | 0.5258 | 0.5333 | 52.7 | 0.0134 | 92 | |
| llmalpha=0.52026.05 | 0.5038 | 0.5026 | 49.7 | 0.0109 | 88.1 |