Code Generation on NaturalCodeBench benchmark leaderboard · SOTA2 Research

Code Generation on NaturalCodeBench

55.7NCB (zh) Python ScoreSeed-Coder-8B-Instruct

Evaluation Results

Method
Seed-Coder-8B-Instruct2026.01	55.7	45.7	50.7	50	47.1	48.6	49.6
Seed-Diffusion-Preview(0705)2026.01	52.9	38.6	45.8	45.7	38.6	42.2	43.9
Stable-DiffCoder-8B-Instruct2026.01	51.4	45.7	48.6	50	47.1	48.6	48.6
Qwen2.5-Coder-14B-Instruct2026.01	48.6	48.6	48.6	42.9	45.7	44.3	46.4
StarCoder2-15B-Instruct2026.01	44.3	30	37.2	38.6	42.9	40.8	39
DeepSeek-Coder-33B-Instruct2026.01	44.3	38.9	41.6	44.3	44.3	44.3	43
Yi-Coder-9B-Chat2026.01	41.4	45.7	43.6	38.6	44.3	41.5	42.5
DeepSeek-Coder-V2-Lite-Instruct2026.01	41.4	47.1	44.3	41.4	37.1	39.3	41.8
OpenCoder-8B-Instruct2026.01	40	30	35	35.7	24.3	30	32.5
Codestral-22B2026.01	40	44.3	42.2	41.4	45.7	43.6	42.9
DeepSeek-Coder-6.7B-Instruct2026.01	38.6	31.4	35	32.9	32.9	32.9	33.9
Qwen3-8B2026.01	37.1	32.9	35	34.3	38.6	36.5	35.7
CodeLlama-70B-Instruct2026.01	35.1	32.1	33.6	32.8	30.5	31.7	32.6
Qwen2.5-Coder-7B-Instruct2026.01	34.3	37.1	35.7	34.3	35.7	35	35.4
CodeQwen1.5-7B-Chat2026.01	30	28.6	29.3	30	27.1	28.6	25.7
Llama-3.1-8B-Instruct2026.01	27.1	24.3	25.7	22.9	22.9	22.9	24.3
CodeLlama-7B-Instruct2026.01	18.6	8.6	13.6	17.1	14.3	15.7	14.6