Fill-in-the-Middle Code Completion on FIM-Tasks Single-Line Infilling (test)
68.3Python ScoreCodeLlama-Base
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CodeLlama-BaseSize=13B2024.01 | 68.3 | 77.6 | 80.7 | 75.5 | |
| CodeLlama-BaseSize=7B2024.01 | 67.6 | 74.3 | 80.2 | 74 | |
| DeepSeek-Coder-BaseSize=7B2024.01 | 66.6 | 88.1 | 79.7 | 78.1 | |
| DeepSeek-Coder-BaseSize=33B2024.01 | 65.4 | 86.6 | 82.5 | 78.2 | |
| StarCoderSize=16B2024.01 | 62 | 73 | 74 | 69.7 | |
| DeepSeek-Coder-BaseSize=1B2024.01 | 57.4 | 82.2 | 71.7 | 70.4 | |
| SantaCoderSize=1.1B2024.01 | 44 | 62 | 74 | 60 |