Code Generation on CodeSearchNet (test) benchmark leaderboard · SOTA2 Research

Code Generation on CodeSearchNet (test)

0.8789BLEUOriginal Prompts

Evaluation Results

Method	Links
Original Prompts2024.09		0.8789	0.9287	0.8869	0.9125	0.9861
TACO-RL2024.09		0.3504	0.6126	0.3883	0.5077	0.9226
TACO-RL2024.09		0.2997	0.5932	0.3399	0.4745	0.9176
TACO-RL2024.09		0.2881	0.5732	0.3399	0.469	0.9156
TACO-RL2024.09		0.2423	0.5489	0.3126	0.4461	0.9114
LLMLingua2024.09		0.2358	0.5523	0.2864	0.4341	0.9112
TACO-RL2024.09		0.2277	0.5439	0.2734	0.4181	0.9083
TACO-RL2024.09		0.2113	0.5267	0.2914	0.4284	0.9081
LLMLingua-22024.09		0.199	0.5293	0.2435	0.3965	0.9061
TACO-RL2024.09		0.1846	0.5099	0.2733	0.4134	0.9052
LLMLingua-22024.09		0.183	0.5157	0.225	0.3794	0.9027
TACO-RL2024.09		0.1759	0.5087	0.2303	0.3848	0.9022
TACO-RL2024.09		0.1415	0.4794	0.202	0.361	0.8976
LLMLingua2024.09		0.134	0.4629	0.1821	0.3449	0.8933
TACO-RL2024.09		0.1238	0.4607	0.1873	0.3493	0.8953
LLMLingua-22024.09		0.1211	0.4494	0.1581	0.321	0.891
LLMLingua-22024.09		0.1198	0.4495	0.1583	0.3207	0.8906
LLMLingua-22024.09		0.0935	0.4162	0.1281	0.2921	0.8845
LLMLingua2024.09		0.0906	0.4151	0.1369	0.3015	0.8834
LLMLingua-22024.09		0.0893	0.4107	0.123	0.2887	0.8831
LLMLingua-22024.09		0.0776	0.3909	0.11	0.2746	0.88
LLMLingua-22024.09		0.0731	0.3888	0.1056	0.2721	0.8787
LLMLingua-22024.09		0.0679	0.3763	0.0995	0.2649	0.8769
LLMLingua2024.09		0.0657	0.3863	0.1113	0.2781	0.8774
LLMLingua-22024.09		0.0654	0.3728	0.0958	0.262	0.8764
LLMLingua2024.09		0.0531	0.3662	0.0976	0.2638	0.8736