List your company

Code Generation on HumanEval (Accuracy Deg %, BERTScore)

-0.37Accuracy Degradation (%)PromptCOS
-0.48160.27171.0251.7783Sep 3, 2025

Evaluation results

MethodLinks
PromptCOSModel=CodeGemma-2b2025.09-0.370.73
PC*Model=CodeGemma-7b2025.090.370.77
PromptCOSModel=CodeGemma-7b2025.090.580.66
PC*Model=CodeGemma-2b2025.091.360.74
PCGModel=CodeGemma-7b2025.091.810.53
PCGModel=CodeGemma-2b2025.092.420.56