Code Completion on MBPP Sanitized
79.9MBPP Pass RateOpenCoder-8B-Base
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OpenCoder-8B-BaseSize=8B2024.11 | 79.9 | 70.4 | |
| Qwen2.5-Coder-7BSize=7B2024.11 | 76.9 | 62.9 | |
| CodeQwen1.5-7BSize=7B2024.11 | 72.2 | 60.2 | |
| DS-Coder-V2-Lite-Base (MoE)Size=16B2024.11 | 71.9 | 59.4 | |
| OpenCoder-1.5B-BaseSize=1.5B2024.11 | 70.6 | 58.7 | |
| DS-Coder-6.7B-BaseSize=6.7B2024.11 | 70.2 | 56.6 | |
| Qwen2.5-Coder-1.5BSize=1.5B2024.11 | 69.2 | 58.6 | |
| StarCoder2-15BSize=15B2024.11 | 66.2 | 53.1 | |
| StarCoder2-3BSize=3B2024.11 | 60.2 | 49.1 | |
| DeepSeek-Coder-1.3B-BaseSize=1.3B2024.11 | 55.6 | 46.9 | |
| CodeLlama-7BSize=7B2024.11 | 55.3 | 46.8 | |
| StarCoder2-7BSize=7B2024.11 | 54.4 | 45.6 | |
| CodeGemma-2BSize=2B2024.11 | 51.1 | 43.1 | |
| CodeGemma-7BSize=7B2024.11 | 50.5 | 40.7 | |
| Yi-Coder-9BSize=9B2024.11 | 48.4 | 40.7 | |
| Crystal-7BSize=7B2024.11 | 38.6 | 31.7 | |
| Yi-Coder-1.5BSize=1.5B2024.11 | 27 | 22.2 |