Security Bug Detection on RepoAudit
2FPMeta Infer
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Meta Infer2026.04 | 2 | 7 | 77.8 | 0 | — | |
| RepoAuditModel=Claude 3.7 Sonnet2026.04 | 5 | 40 | 88.9 | — | 23.85 | |
| RepoAuditModel=DeepSeek R12026.04 | 6 | 41 | 87.2 | — | 8.55 | |
| SPECAModel=Sonnet 4.52026.04 | 6 | — | 88.9 | 12 | 81.05 | |
| RepoAuditModel=o3-mini2026.04 | 9 | 36 | 80 | — | 4.5 | |
| RepoAuditModel=Claude 3.5 Sonnet2026.04 | 11 | 40 | 78.4 | — | 38.1 | |
| SPECAModel=Sonnet 42026.04 | 13 | — | 81.2 | 18 | 100.68 | |
| SPECAModel=DeepSeek R12026.04 | 15 | — | 72.7 | 7 | 93.51 | |
| Amazon CodeGuru2026.04 | 18 | 0 | 0 | 0 | — |