Security Smell Detection on Puppet Oracle (test)
84.6PrecisionIntelliSA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IntelliSA2026.01 | 84.6 | 84.6 | 84.6 | |
| Grok-4model=Grok-4-fast, Prompting style=static-analysis-rules2026.01 | 71.8 | 78.5 | 75 | |
| Grok-4model=Grok-4-fast, Prompting style=definition-based2026.01 | 66.7 | 73.8 | 70.1 | |
| Claude-4model=Claude-Sonnet-4.0, Prompting style=definition-based2026.01 | 62.3 | 73.8 | 67.6 | |
| GPT-5model=GPT-5-2025-08-07, Prompting style=definition-based2026.01 | 59.3 | 53.8 | 56.5 | |
| GPT-5model=GPT-5-2025-08-07, Prompting style=static-analysis-rules2026.01 | 54.2 | 69.2 | 60.8 | |
| Claude-4model=Claude-Sonnet-4.0, Prompting style=static-analysis-rules2026.01 | 50 | 61.5 | 55.2 | |
| GLITCH2026.01 | 41.8 | 93.8 | 57.8 | |
| SLAC/SLIC2026.01 | 27.5 | 50.8 | 35.7 |