Security-aware code generation on Asleep at the Keyboard DoW
98.7ValidityStarCoderBase
Evaluation results
| Method | Links | ||
|---|---|---|---|
| StarCoderBaseFormat=Insertion2023.05 | 98.7 | 35.87 | |
| code-cushman-001Format=Completion2023.05 | 96.4 | 42.32 | |
| CodeGen-16B-MultiFormat=Completion2023.05 | 95.5 | 43.25 | |
| InCoder-6BFormat=Completion2023.05 | 87.1 | 35.48 | |
| StarCoderBaseFormat=Completion2023.05 | 85.5 | 39.77 | |
| InCoder-6BFormat=Insertion2023.05 | 85.4 | 34.31 |