Commit Message Generation on Commit Chronicle
15.88ROUGE-2 ScoreIRCoder
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| IRCoderParameters=7.3B2024.03 | 15.88 | 38.96 | |
| StarCoderBaseParameters=7.3B2024.03 | 15.22 | 37.74 | |
| IRCoderParameters=6.7B2024.03 | 14.95 | 36.82 | |
| IRCoderParameters=5.7B2024.03 | 14.71 | 37.15 | |
| CodeLlamaParameters=6.7B2024.03 | 14.46 | 35.91 | |
| StarCoderBaseParameters=3.1B2024.03 | 14.35 | 33.7 | |
| IRCoderParameters=3.1B2024.03 | 14.29 | 36.81 | |
| DeepSeekCoderParameters=5.7B2024.03 | 13.77 | 35.18 | |
| IRCoderParameters=1.1B2024.03 | 13.15 | 35.04 | |
| IRCoderParameters=1.3B2024.03 | 13.12 | 34.57 | |
| StarCoderBaseParameters=1.1B2024.03 | 12.41 | 33.67 | |
| DeepSeekCoderParameters=1.3B2024.03 | 12.33 | 33.16 |