Code Generation on HumanEval HE, HE+
88.4HE Pass RateQwen2.5-Coder-7B-Instruct
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen2.5-Coder-7B-InstructSize=7B2024.11 | 88.4 | 84.1 | |
| CodeQwen1.5-7B-ChatSize=7B2024.11 | 86 | 79.3 | |
| OpenCoder-8B-InstructSize=8B2024.11 | 83.5 | 78.7 | |
| Yi-Coder-9B-ChatSize=9B2024.11 | 82.3 | 72.6 | |
| DS-Coder-V2-Lite-InstructSize=16B2024.11 | 81.1 | 75 | |
| DS-Coder-6.7B-InstructSize=6.7B2024.11 | 78.6 | 70.7 | |
| StarCoder2-15B-Instruct-v0.1Size=15B2024.11 | 72.6 | 63.4 | |
| OpenCoder-1.5B-InstructSize=1.5B2024.11 | 72.5 | 67.7 | |
| Qwen2.5-Coder-1.5B-InstructSize=1.5B2024.11 | 70.7 | 66.5 | |
| Yi-Coder-1.5B-ChatSize=1.5B2024.11 | 67.7 | 63.4 | |
| DS-coder-1.3B-InstructSize=1.3B2024.11 | 65.2 | 61.6 | |
| CodeGemma-7B-ItSize=7B2024.11 | 59.8 | 47 | |
| CodeLlama-7B-InstructSize=7B2024.11 | 45.7 | 39.6 | |
| CrystalChat-7BSize=7B2024.11 | 34.1 | 31.7 |