Fill-in-the-middle on FIM-task Java v1.0 (test)
81.1Exact MatchStarCoder2-7B
Evaluation results
| Method | Links | |
|---|---|---|
| StarCoder2-7BModel Size=7B2024.02 | 81.1 | |
| CodeLlama-13BModel Size=13B2024.02 | 80 | |
| StarCoder2-3BModel Size=3B2024.02 | 75 | |
| StarCoder Base-15BModel Size=15B2024.02 | 73 | |
| StableCode-3BModel Size=3B2024.02 | 63.7 | |
| StarCoder2-15BModel Size=15B, Note=Implementation bug during training affected FIM performance2024.02 | 60.5 |