Code Generation on ReCode Function
38.66pass@1IRCoder
Evaluation Results
| Method | Links | |
|---|---|---|
| IRCoderModel Size=5.7B2024.03 | 38.66 | |
| DeepSeekCoderModel Size=5.7B2024.03 | 36.73 | |
| IRCoderModel Size=1.3B2024.03 | 25.75 | |
| IRCoderModel Size=6.7B2024.03 | 25.47 | |
| DeepSeekCoderModel Size=1.3B2024.03 | 25.13 | |
| CodeLlamaModel Size=6.7B2024.03 | 24.49 | |
| IRCoderModel Size=7.3B2024.03 | 23.76 | |
| StarCoderBaseModel Size=7.3B2024.03 | 23.53 | |
| StarCoderBaseModel Size=3.1B2024.03 | 19.04 | |
| IRCoderModel Size=3.1B2024.03 | 18.8 | |
| IRCoderModel Size=1.1B2024.03 | 12.01 | |
| StarCoderBaseModel Size=1.1B2024.03 | 11.31 |