Fill-in-the-middle on FIM-task JavaScript v1.0 (test)
85Exact MatchCodeLlama-13B
Evaluation results
| Method | Links | |
|---|---|---|
| CodeLlama-13BModel Size=13B2024.02 | 85 | |
| StarCoder2-7BModel Size=7B2024.02 | 77.5 | |
| StarCoder Base-15BModel Size=15B2024.02 | 74 | |
| StableCode-3BModel Size=3B2024.02 | 73.3 | |
| StarCoder2-3BModel Size=3B2024.02 | 73 | |
| StarCoder2-15BModel Size=15B, Note=Implementation bug during training affected FIM performance2024.02 | 54.7 |