Insecure Coding Practice on CyberSecEval Autocomplete 1
85.9CLlama2-7b-chat
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Llama2-7b-chatModel size=7b2024.06 | 85.9 | 93.2 | 93.1 | 88.7 | 93.6 | 88.9 | 76.4 | 90.7 | 88.1 | |
| CommandR+INDICT2024.06 | 83.7 | 96.7 | 90.3 | 89.7 | 91.9 | 88.8 | 81.9 | 75.4 | 87.2 | |
| Llama2-13b-chatModel size=13b2024.06 | 77.5 | 90.6 | 84.2 | 76.4 | 91.6 | 81.5 | 72.9 | 85.8 | 82.1 | |
| Llama2-30b-chatModel size=30b2024.06 | 71.8 | 84.3 | 84.6 | 68.1 | 84.7 | 82.1 | 74.1 | 86.8 | 79.2 | |
| Codellama-13b-instructModel size=13b2024.06 | 70 | 83.4 | 79.5 | 70.7 | 81.5 | 75.9 | 70.7 | 76 | 75.8 | |
| CommandR2024.06 | 70 | 92.6 | 82.9 | 77.6 | 87.8 | 82.8 | 64.9 | 71.9 | 78.4 | |
| CommandR+INDICTFramework=INDICT2024.06 | 68.2 | 67.2 | 64.2 | 66.3 | 66.3 | 70.4 | 69.2 | 65.5 | 67.2 | |
| Llama2-70b-chatModel size=70b2024.06 | 67 | 75.3 | 87.3 | 71.6 | 85.9 | 80.9 | 67.2 | 78.4 | 76.2 | |
| GPT-3.5-turbo2024.06 | 66.5 | 83 | 79.2 | 63.3 | 77.5 | 77.2 | 59 | 63.2 | 70.5 | |
| Codellama-34b-instructModel size=34b2024.06 | 65.6 | 81.3 | 78.4 | 69 | 77.5 | 76.5 | 66.1 | 70.6 | 72.8 | |
| GPT-42024.06 | 61.2 | 70.6 | 75.3 | 59.4 | 65.5 | 71 | 49.9 | 62.8 | 63.5 | |
| GPT42024.06 | 57.3 | 65.5 | 60.6 | 63.8 | 52.4 | 55.9 | 59.8 | 64.2 | 60 | |
| CommandR2024.06 | 54.7 | 54 | 49.5 | 52.8 | 47.3 | 45.1 | 45.1 | 50.6 | 49.7 | |
| GPT3.52024.06 | 44.9 | 41.3 | 44.4 | 56.3 | 36.7 | 37.9 | 40.7 | 44.1 | 43.3 |