Retrieval-Augmented Code Generation on CSN-Python Target Queries (test)
41.44Verification Rate (GPT-4o)VenomRACG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VenomRACGSetting=Black-box, Target=given (7.77%)2025.12 | 41.44 | 38.71 | |
| VenomRACGSetting=White-box, Target=file (6.28%)2025.12 | 39.27 | 42.21 | |
| VenomRACGSetting=White-box, Target=given (7.77%)2025.12 | 35.98 | 36.27 | |
| VenomRACGSetting=White-box, Target=get (7.99%)2025.12 | 33.91 | 32.35 | |
| VenomRACGSetting=Black-box, Target=file (6.28%)2025.12 | 29.51 | 24.41 | |
| VenomRACGSetting=Black-box, Target=get (7.99%)2025.12 | 29.49 | 32.87 | |
| BadCodeSetting=White-box, Target=get (7.99%)2025.12 | 26.37 | 22.03 | |
| DeadCodeSetting=White-box, Target=get (7.99%)2025.12 | 25.5 | 19.08 | |
| BadCodeSetting=Black-box, Target=get (7.99%)2025.12 | 24.5 | 20.21 | |
| DeadCodeSetting=Black-box, Target=get (7.99%)2025.12 | 23.67 | 18.99 | |
| DeadCodeSetting=White-box, Target=given (7.77%)2025.12 | 23.45 | 20.84 | |
| DeadCodeSetting=Black-box, Target=file (6.28%)2025.12 | 22.99 | 22.42 | |
| BadCodeSetting=White-box, Target=file (6.28%)2025.12 | 21.82 | 22.04 | |
| BadCodeSetting=Black-box, Target=file (6.28%)2025.12 | 21.75 | 20.75 | |
| DeadCodeSetting=Black-box, Target=given (7.77%)2025.12 | 21.53 | 24.26 | |
| BadCodeSetting=White-box, Target=given (7.77%)2025.12 | 21.24 | 18.39 | |
| BadCodeSetting=Black-box, Target=given (7.77%)2025.12 | 20.71 | 22.62 | |
| DeadCodeSetting=White-box, Target=file (6.28%)2025.12 | 18.22 | 27.84 | |
| No AttackSetting=No Poisoned, Target=N/A2025.12 | 17.56 | 7.56 |