Vulnerability Repair Evaluation on 96-repair corpus (test)
89.6Targeted Removal RateGPT-4o
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| GPT-4oOracle evaluation=true, Number of repairs=962026.06 | 89.6 | 8.3 | 93.8 | 43.8 | 42.7 | 64.3 | 100 | 80 | |
| Claude 3.5 SonnetOracle evaluation=true, Number of repairs=962026.06 | 87.5 | 12.5 | 95.8 | 48 | 47 | 57.1 | 95.7 | 85.7 | |
| Gemini-2.5-flash-liteOracle evaluation=true, Number of repairs=962026.06 | 83.3 | 10.4 | 90.6 | 39.6 | 38.5 | 71.4 | 100 | 90 |