Code Completion on MBPP (test)
69.43-Shot Pass RateYi-Coder-9B
Evaluation Results
| Method | Links | |
|---|---|---|
| Yi-Coder-9BSize=9B2024.11 | 69.4 | |
| Qwen2.5-Coder-7BSize=7B2024.11 | 68.8 | |
| DS-Coder-V2-Lite-Base (MoE)Size=16B2024.11 | 62.6 | |
| CodeQwen1.5-7BSize=7B2024.11 | 61.8 | |
| DS-Coder-6.7B-BaseSize=6.7B2024.11 | 60.6 | |
| OpenCoder-8B-BaseSize=8B2024.11 | 60.6 | |
| Qwen2.5-Coder-1.5BSize=1.5B2024.11 | 59.2 | |
| StarCoder2-7BSize=7B2024.11 | 55.2 | |
| CodeGemma-7BSize=7B2024.11 | 55 | |
| OpenCoder-1.5B-BaseSize=1.5B2024.11 | 51.8 | |
| Yi-Coder-1.5BSize=1.5B2024.11 | 51.6 | |
| StarCoder2-3BSize=3B2024.11 | 46.4 | |
| DeepSeek-Coder-1.3B-BaseSize=1.3B2024.11 | 46.2 | |
| CodeGemma-2BSize=2B2024.11 | 45.4 | |
| CodeLlama-7BSize=7B2024.11 | 41.4 | |
| Crystal-7BSize=7B2024.11 | 31 |