Code Generation on ReCode Syntax
59.24Pass@1IRCoder
Evaluation results
| Method | Links | |
|---|---|---|
| IRCoderModel Size=5.7B2024.03 | 59.24 | |
| DeepSeekCoderModel Size=5.7B2024.03 | 55.43 | |
| IRCoderModel Size=6.7B2024.03 | 48.11 | |
| IRCoderModel Size=1.3B2024.03 | 45.43 | |
| CodeLlamaModel Size=6.7B2024.03 | 45.23 | |
| DeepSeekCoderModel Size=1.3B2024.03 | 44.88 | |
| IRCoderModel Size=7.3B2024.03 | 41.82 | |
| StarCoderBaseModel Size=7.3B2024.03 | 41.5 | |
| IRCoderModel Size=3.1B2024.03 | 34.42 | |
| StarCoderBaseModel Size=3.1B2024.03 | 33.29 | |
| IRCoderModel Size=1.1B2024.03 | 27.5 | |
| StarCoderBaseModel Size=1.1B2024.03 | 26.39 |