Code Infilling on HumanEval random span
74.2pass@1 (PSM)code-davinci-002
Evaluation results
| Method | Links | ||
|---|---|---|---|
| code-davinci-002Size=175B, Temperature=0.1, Decoding Strategy=nucleus sampling (p=0.95)2023.08 | 74.2 | — | |
| Code LlamaSize=13B, LCFT=false, Decoding Strategy=greedy decoding2023.08 | 64.3 | 42.7 | |
| Code LlamaSize=13B, LCFT=true, Decoding Strategy=greedy decoding2023.08 | 63.6 | 41.9 | |
| Code LlamaSize=7B, LCFT=false, Decoding Strategy=greedy decoding2023.08 | 60.7 | 39.6 | |
| Code LlamaSize=7B, LCFT=true, Decoding Strategy=greedy decoding2023.08 | 59.7 | 39 | |
| OpenAI FIM90Size=7B, Temperature=0.1, Decoding Strategy=nucleus sampling (p=0.95)2023.08 | 55.1 | — |