Code Generation on EvoEval semantic-altering benchmark leaderboard · SOTA2 Research

Code Generation on EvoEval semantic-altering

48Pass@1 AverageCodeQwen1.5-7B-Chat

Evaluation Results

Method	Links
CodeQwen1.5-7B-Chat2024.10		48	39	38	71	31	61
SelfCodeAlign-CQ-7B2024.10		43.6	33	40	60	20	65
CodeQwen1.5-7B-OctoPack2024.10		42.2	35	36	59	22	59
DeepSeek-Coder-6.7B-Instruct2024.10		41.4	40	37	61	18	51
Llama-3-8B-Instruct2024.10		40.6	34	39	57	15	58
CodeQwen1.5-7B-Base2024.10		36.2	26	30	46	18	61
CodeGemma-7B-IT2024.10		35.4	31	32	49	9	56
OctoCoder-16B2024.10		30.6	19	26	43	11	54
StarCoder2-15B2024.10		25.8	16	19	41	5	48