Code Completion on BigCodeBench
45.8Full ScoreQwen2.5-Coder-7B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Qwen2.5-Coder-7BSize=7B2024.11 | 45.8 | 16.2 | |
| CodeQwen1.5-7BSize=7B2024.11 | 45.6 | 15.6 | |
| Yi-Coder-9BSize=9B2024.11 | 42.9 | 14.2 | |
| DS-Coder-6.7B-BaseSize=6.7B2024.11 | 41.1 | 11.5 | |
| OpenCoder-8B-BaseSize=8B2024.11 | 40.5 | 9.5 | |
| StarCoder2-15BSize=15B2024.11 | 38.4 | 12.2 | |
| CodeGemma-7BSize=7B2024.11 | 38.3 | 10.1 | |
| Qwen2.5-Coder-1.5BSize=1.5B2024.11 | 34.6 | 9.5 | |
| DS-Coder-V2-Lite-Base (MoE)Size=16B2024.11 | 30.6 | 8.1 | |
| CodeLlama-7BSize=7B2024.11 | 28.7 | 5.4 | |
| StarCoder2-7BSize=7B2024.11 | 27.7 | 8.8 | |
| DeepSeek-Coder-1.3B-BaseSize=1.3B2024.11 | 26.1 | 3.4 | |
| OpenCoder-1.5B-BaseSize=1.5B2024.11 | 24.5 | 5.4 | |
| CodeGemma-2BSize=2B2024.11 | 23.9 | 7.4 | |
| Yi-Coder-1.5BSize=1.5B2024.11 | 23.5 | 3.4 | |
| StarCoder2-3BSize=3B2024.11 | 21.4 | 4.7 | |
| Crystal-7BSize=7B2024.11 | 10.8 | 4.1 |