Code Generation on MBPP (Pass%, CodeBLEU) benchmark leaderboard · SOTA2 Research

Code Generation on MBPP (Pass%, CodeBLEU)

99.78Pass%Iterative

Evaluation Results

Method	Links
Iterative2026.06		99.78	1.2	0.974	70.7	89.1	0.46
Refactoring-Based2026.06		99.67	—	0.723	8	8	0.65
Refactoring-Based2026.06		99.45	—	0.879	11.1	42.2	0.91
Static2026.06		98.25	—	0.922	39.8	59.7	0.44
Iterative2026.06		97.27	1.33	0.989	90	96	0.46
Iterative-AGL2026.06		96.79	1.9	0.994	96.7	97.9	0.46
Comment-based2026.06		95.01	—	0.925	41.8	68.6	0.44
ICW2026.06		94.44	—	0.512	4.5	7.2	0.44
Iterative-AGL2026.06		93.6	2.5	0.987	87	94	0.47
ICW2026.06		92.4	—	0.77	2	14	0.5
Comment-based2026.06		90	—	0.892	39	60	0.47
Static2026.06		86.73	—	0.899	26	56	0.46
No Watermark2026.06		64.77	—	0.577	3.5	10.6	0.42
No Watermark2026.06		63.96	—	0.416	4	6	0.5
SWEET2026.06		43.83	—	0.92	44	72	0.42