Code Generation on BigCodeBench-Completion Full
59.7pass@1DeepSeek-Coder-V2-Instruct
Evaluation results
| Method | Links | |
|---|---|---|
| DeepSeek-Coder-V2-InstructSize=21B/236B2026.01 | 59.7 | |
| DS-Coder-V2-InstructSize Category=32B+ LLMs2025.12 | 59.7 | |
| Claude-3.5-Sonnet-20241022Size Category=Closed-APIs2025.12 | 58.6 | |
| Qwen2.5-Coder-32B-InstructSize Category=32B+ LLMs2025.12 | 58 | |
| Qwen2.5-Coder-14B-InstructSize Category=13B+ LLMs2025.12 | 56.7 | |
| UCoder-32BSize Category=32B+ LLMs2025.12 | 55.4 | |
| Stable-DiffCoder-8B-InstructSize=8B2026.01 | 54.8 | |
| UCoder-14BSize Category=13B+ LLMs2025.12 | 53.9 | |
| Qwen2.5-Coder-32BSize Category=32B+ LLMs2025.12 | 53.6 | |
| Seed-Coder-8B-InstructSize=8B2026.01 | 53.3 | |
| Seed-Diffusion-Preview(0705)2026.01 | 53.2 | |
| Codestral-22BSize=22B2026.01 | 52.5 | |
| Qwen2.5-Coder-32B-InstructSize=32B2026.01 | 52.3 | |
| Qwen2.5-Coder-14B-InstructSize=14B2026.01 | 52.2 | |
| UCoder-7BSize Category=6B+ LLMs2025.12 | 52 | |
| Qwen2.5-Coder-14BSize Category=13B+ LLMs2025.12 | 51.8 | |
| Qwen3-8BSize=8B2026.01 | 51.7 | |
| DeepSeek-Coder-33B-InstructSize=33B2026.01 | 51.1 | |
| DS-Coder-33B-InstructSize Category=32B+ LLMs2025.12 | 51.1 | |
| OpenCoder-8B-InstructSize=8B2026.01 | 50.9 | |
| OpenCoder-8B-InstructSize Category=6B+ LLMs2025.12 | 50.9 | |
| CodeLlama-70B-InstructSize=70B2026.01 | 49.6 | |
| Yi-Coder-9B-ChatSize=9B2026.01 | 49 | |
| Qwen2.5-Coder-7B-InstructSize=7B2026.01 | 48.8 | |
| Qwen2.5-Coder-7B-InstructSize Category=6B+ LLMs2025.12 | 48.8 | |
| DeepSeek-Coder-V2-Lite-InstructSize=2.4B/16B2026.01 | 47.6 | |
| DS-Coder-V2-Lite-InstructSize Category=13B+ LLMs2025.12 | 47.6 | |
| Qwen2.5-Coder-7BSize Category=6B+ LLMs2025.12 | 45.8 | |
| Mercury Coder2026.01 | 45.5 | |
| Gemini Diffusion2026.01 | 45.4 | |
| StarCoder2-15B-InstructSize=15B2026.01 | 45.1 | |
| Starcoder2-15B-Instruct-v0.1Size Category=13B+ LLMs2025.12 | 45.1 | |
| DeepSeek-Coder-6.7B-InstructSize=6.7B2026.01 | 43.8 | |
| DS-Coder-6.7B-InstructSize Category=6B+ LLMs2025.12 | 43.8 | |
| CodeQwen1.5-7B-ChatSize=7B2026.01 | 43.6 | |
| LLaDA2.0-flashSize=6B/100B2026.01 | 41.6 | |
| Llama-3.1-8B-InstructSize=8B2026.01 | 40.5 | |
| CodeLlama-34B-InstructSize Category=32B+ LLMs2025.12 | 35.6 | |
| CodeLlama-13B-InstructSize Category=13B+ LLMs2025.12 | 31.7 | |
| CodeLlama-7B-InstructSize=7B2026.01 | 25.7 | |
| CodeLlama-7B-InstructSize Category=6B+ LLMs2025.12 | 25.7 |