Software Vulnerability Detection on Juliet Suite DBZ (test)
100PrecisionLLMDFA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| LLMDFALLM=gpt-3.5, Phase=Extract2024.02 | 100 | 100 | 100 | |
| LLMDFALLM=gpt-4, Phase=Extract2024.02 | 100 | 100 | 100 | |
| LLMDFALLM=gemini-1.0, Phase=Extract2024.02 | 100 | 100 | 100 | |
| LLMDFALLM=claude-3, Phase=Extract2024.02 | 100 | 100 | 100 | |
| LLMDFALLM=gpt-4, Phase=Summarize2024.02 | 95.32 | 98.43 | 97 | |
| LLMDFALLM=gpt-3.5, Phase=Summarize2024.02 | 90.95 | 97.57 | 94 | |
| LLMDFALLM=gpt-4, Phase=Validate2024.02 | 89.76 | 100 | 95 | |
| LLMDFALLM=claude-3, Phase=Summarize2024.02 | 89.26 | 92.38 | 91 | |
| LLMDFALLM=claude-3, Phase=Validate2024.02 | 85.74 | 94.52 | 90 | |
| LLMDFALLM=gemini-1.0, Phase=Summarize2024.02 | 83.57 | 82.47 | 83 | |
| LLMDFALLM=gpt-3.5, Phase=Validate2024.02 | 81.58 | 99.2 | 90 | |
| LLMDFALLM=gpt-4, Phase=Detection2024.02 | 81.38 | 95.75 | 87 | |
| LLMDFALLM=gemini-1.0, Phase=Validate2024.02 | 79.83 | 93.73 | 86 | |
| LLMDFALLM=claude-3, Phase=Detection2024.02 | 76.91 | 82.67 | 80 | |
| LLMDFALLM=gpt-3.5, Phase=Detection2024.02 | 73.75 | 92.16 | 82 | |
| LLMDFALLM=gemini-1.0, Phase=Detection2024.02 | 66.57 | 74.21 | 70 |