Code Generation on ClassEval
27Pass@1 (Class-level)DeepSeek-Coder-6.7B-Instruct
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeepSeek-Coder-6.7B-Instructdecoding_strategy=greedy2024.10 | 27 | 57.2 | |
| CodeQwen1.5-7B-Chatdecoding_strategy=greedy2024.10 | 27 | 54.6 | |
| SelfCodeAlign-CQ-7Bdecoding_strategy=greedy2024.10 | 27 | 52.6 | |
| Llama-3-8B-Instructdecoding_strategy=greedy2024.10 | 23 | 52.4 | |
| CodeQwen1.5-7B-Basedecoding_strategy=greedy2024.10 | 23 | 52.8 | |
| CodeGemma-7B-ITdecoding_strategy=greedy2024.10 | 21 | 44.8 | |
| CodeQwen1.5-7B-OctoPackdecoding_strategy=greedy2024.10 | 21 | 45.2 | |
| OctoCoder-16Bdecoding_strategy=greedy2024.10 | 19 | 38 | |
| StarCoder2-15Bdecoding_strategy=greedy2024.10 | 9 | 24.9 |