Security Smell Detection on Chef oracle (test)
87.7PrecisionIntelliSA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IntelliSA2026.01 | 87.7 | 68.3 | 76.8 | |
| Grok-4model=Grok-4-fast, Prompting style=static-analysis-rules2026.01 | 66.7 | 63.5 | 65 | |
| GPT-5model=GPT-5-2025-08-07, Prompting style=definition-based2026.01 | 65.5 | 34.6 | 45.3 | |
| Grok-4model=Grok-4-fast, Prompting style=definition-based2026.01 | 62.5 | 57.7 | 60 | |
| GLITCH2026.01 | 60.3 | 73.1 | 66.1 | |
| GPT-5model=GPT-5-2025-08-07, Prompting style=static-analysis-rules2026.01 | 55.4 | 59.6 | 57.4 | |
| Claude-4model=Claude-Sonnet-4.0, Prompting style=definition-based2026.01 | 49 | 49 | 49 | |
| Claude-4model=Claude-Sonnet-4.0, Prompting style=static-analysis-rules2026.01 | 45.7 | 60.6 | 52.1 | |
| SLAC/SLIC2026.01 | 36.7 | 38.5 | 37.6 |