Code Generation on BIRD-Python Original (dev) benchmark leaderboard · SOTA2 Research

Code Generation on BIRD-Python Original (dev)

0.6584Execution Accuracy (Simple)Qwen3 (14B)

Evaluation Results

Method	Links
Qwen3 (14B)2026.01		0.6584	0.3233	0.2966	0.4289
DeepSeek-R12026.01		0.6238	0.4504	0.4828	0.5574
Qwen3-Max2026.01		0.6227	0.4375	0.4759	0.5528
Qwen3-Coder-Plus2026.01		0.6032	0.4526	0.4207	0.5398
Qwen3 (32B)2026.01		0.6022	0.4224	0.4552	0.5332
Qwen-Max2026.01		0.5957	0.4504	0.4276	0.5359
Qwen2.5-Coder (32B)2026.01		0.5838	0.4181	0.3793	0.5143
Qwen3 (7B)2026.01		0.547	0.3621	0.3448	0.472
Qwen2.5-Coder (14B)2026.01		0.5124	0.3384	0.3586	0.4452
Qwen3-Coder (30B)2026.01		0.4692	0.3405	0.3241	0.4153
Qwen2.5-Coder (7B)2026.01		0.3989	0.1897	0.2345	0.3194
TableGPT2 (7B)2026.01		0.2315	0.1509	0.1103	0.1745
AutoCoder (7B)2026.01		0.075	0.0388	0.0172	0.0488
StarCoder2 (7B)2026.01		0.0162	0.0043	0	0.0085