Code Generation on ReCode Format
65.76Pass@1IRCoder
Evaluation results
| Method | Links | |
|---|---|---|
| IRCoderModel Size=5.7B2024.03 | 65.76 | |
| DeepSeekCoderModel Size=5.7B2024.03 | 62.37 | |
| IRCoderModel Size=6.7B2024.03 | 56.41 | |
| CodeLlamaModel Size=6.7B2024.03 | 54.5 | |
| IRCoderModel Size=1.3B2024.03 | 49.85 | |
| DeepSeekCoderModel Size=1.3B2024.03 | 49.61 | |
| IRCoderModel Size=7.3B2024.03 | 46.62 | |
| StarCoderBaseModel Size=7.3B2024.03 | 46.3 | |
| IRCoderModel Size=3.1B2024.03 | 39.78 | |
| StarCoderBaseModel Size=3.1B2024.03 | 38.7 | |
| IRCoderModel Size=1.1B2024.03 | 30.18 | |
| StarCoderBaseModel Size=1.1B2024.03 | 28.08 |