Code Helpfulness Evaluation on CVS (test) benchmark leaderboard · SOTA2 Research

Code Helpfulness Evaluation on CVS (test)

0.66C++ Success RateLlama3-70b-instruct

Evaluation Results

Method	Links
Llama3-70b-instruct2024.06		0.66	0.57	0.67	0.69	0.7	0.658
Llama3-8b-instruct2024.06		0.62	0.58	0.6	0.64	0.68	0.624
Llama3-70b-instruct2024.06		0.52	0.55	0.53	0.53	0.42	0.51
GT Secure Code2024.06		0.5	0.5	0.5	0.5	0.5	0.5
Llama3-8b-instruct2024.06		0.5	0.5	0.49	0.49	0.39	0.474
CommandR2024.06		0.38	0.38	0.42	0.515	0.43	0.425
GT UnSecure Code2024.06		0.35	0.33	0.4	0.52	0.44	0.408
CommandR2024.06		0.28	0.61	0.63	0.596	0.74	0.571