Security Smell Detection on Ansible oracle (test)
94.7PrecisionIntelliSA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IntelliSA2026.01 | 94.7 | 81.8 | 87.8 | |
| Grok-4model=Grok-4-fast, Prompting style=definition-based2026.01 | 85.7 | 68.2 | 75.9 | |
| Grok-4model=Grok-4-fast, Prompting style=static-analysis-rules2026.01 | 83.3 | 90.9 | 87 | |
| GLITCH2026.01 | 74.1 | 97.7 | 84.3 | |
| GPT-5model=GPT-5-2025-08-07, Prompting style=definition-based2026.01 | 74.1 | 45.5 | 56.3 | |
| GPT-5model=GPT-5-2025-08-07, Prompting style=static-analysis-rules2026.01 | 67.8 | 90.9 | 77.7 | |
| Claude-4model=Claude-Sonnet-4.0, Prompting style=definition-based2026.01 | 61 | 81.8 | 69.9 | |
| Claude-4model=Claude-Sonnet-4.0, Prompting style=static-analysis-rules2026.01 | 50.8 | 75 | 60.6 | |
| SLAC/SLIC2026.01 | 47.1 | 54.5 | 50.5 |