Code Generation on NaturalCodeBench
55.7NCB (zh) Python ScoreSeed-Coder-8B-Instruct
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Seed-Coder-8B-InstructSize=8B2026.01 | 55.7 | 45.7 | 50.7 | 50 | 47.1 | 48.6 | 49.6 | |
| Seed-Diffusion-Preview(0705)Size=-2026.01 | 52.9 | 38.6 | 45.8 | 45.7 | 38.6 | 42.2 | 43.9 | |
| Stable-DiffCoder-8B-InstructSize=8B2026.01 | 51.4 | 45.7 | 48.6 | 50 | 47.1 | 48.6 | 48.6 | |
| Qwen2.5-Coder-14B-InstructSize=14B2026.01 | 48.6 | 48.6 | 48.6 | 42.9 | 45.7 | 44.3 | 46.4 | |
| StarCoder2-15B-InstructSize=15B2026.01 | 44.3 | 30 | 37.2 | 38.6 | 42.9 | 40.8 | 39 | |
| DeepSeek-Coder-33B-InstructSize=33B2026.01 | 44.3 | 38.9 | 41.6 | 44.3 | 44.3 | 44.3 | 43 | |
| Yi-Coder-9B-ChatSize=9B2026.01 | 41.4 | 45.7 | 43.6 | 38.6 | 44.3 | 41.5 | 42.5 | |
| DeepSeek-Coder-V2-Lite-InstructSize=2.4B/16B2026.01 | 41.4 | 47.1 | 44.3 | 41.4 | 37.1 | 39.3 | 41.8 | |
| OpenCoder-8B-InstructSize=8B2026.01 | 40 | 30 | 35 | 35.7 | 24.3 | 30 | 32.5 | |
| Codestral-22BSize=22B2026.01 | 40 | 44.3 | 42.2 | 41.4 | 45.7 | 43.6 | 42.9 | |
| DeepSeek-Coder-6.7B-InstructSize=6.7B2026.01 | 38.6 | 31.4 | 35 | 32.9 | 32.9 | 32.9 | 33.9 | |
| Qwen3-8BSize=8B2026.01 | 37.1 | 32.9 | 35 | 34.3 | 38.6 | 36.5 | 35.7 | |
| CodeLlama-70B-InstructSize=70B2026.01 | 35.1 | 32.1 | 33.6 | 32.8 | 30.5 | 31.7 | 32.6 | |
| Qwen2.5-Coder-7B-InstructSize=7B2026.01 | 34.3 | 37.1 | 35.7 | 34.3 | 35.7 | 35 | 35.4 | |
| CodeQwen1.5-7B-ChatSize=7B2026.01 | 30 | 28.6 | 29.3 | 30 | 27.1 | 28.6 | 25.7 | |
| Llama-3.1-8B-InstructSize=8B2026.01 | 27.1 | 24.3 | 25.7 | 22.9 | 22.9 | 22.9 | 24.3 | |
| CodeLlama-7B-InstructSize=7B2026.01 | 18.6 | 8.6 | 13.6 | 17.1 | 14.3 | 15.7 | 14.6 |