Software Vulnerability Detection on Juliet Suite OSCI (test)
1PrecisionLLMDFA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| LLMDFALLM=gpt-3.5, Phase=Extract2024.02 | 1 | 1 | 1 | |
| LLMDFALLM=gpt-3.5, Phase=Validate2024.02 | 1 | 0.9714 | 0.99 | |
| LLMDFALLM=gpt-3.5, Phase=Detection2024.02 | 1 | 0.7838 | 0.88 | |
| LLMDFALLM=gpt-4, Phase=Extract2024.02 | 1 | 1 | 1 | |
| LLMDFALLM=gpt-4, Phase=Validate2024.02 | 1 | 1 | 1 | |
| LLMDFALLM=gpt-4, Phase=Detection2024.02 | 1 | 0.8919 | 0.94 | |
| LLMDFALLM=gemini-1.0, Phase=Extract2024.02 | 1 | 1 | 1 | |
| LLMDFALLM=gemini-1.0, Phase=Validate2024.02 | 1 | 0.9813 | 0.99 | |
| LLMDFALLM=gemini-1.0, Phase=Detection2024.02 | 1 | 0.9459 | 0.97 | |
| LLMDFALLM=claude-3, Phase=Extract2024.02 | 1 | 1 | 1 | |
| LLMDFALLM=claude-3, Phase=Validate2024.02 | 1 | 1 | 1 | |
| LLMDFALLM=claude-3, Phase=Detection2024.02 | 1 | 0.973 | 0.99 | |
| LLMDFALLM=claude-3, Phase=Summarize2024.02 | 0.9826 | 0.9787 | 0.98 | |
| LLMDFALLM=gpt-4, Phase=Summarize2024.02 | 0.9458 | 0.9312 | 0.94 | |
| LLMDFALLM=gpt-3.5, Phase=Summarize2024.02 | 0.8957 | 0.8576 | 0.88 | |
| LLMDFALLM=gemini-1.0, Phase=Summarize2024.02 | 0.8721 | 0.9654 | 0.92 |