Retrieval-Augmented Code Generation on CSN-Python Non-target Queries (test)
0.318Similarity (GPT-4o)VenomRACG
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| VenomRACGSetting=Black-box, Target=given (7.77%)2025.12 | 0.318 | 0.487 | |
| BadCodeSetting=Black-box, Target=given (7.77%)2025.12 | 0.315 | 0.528 | |
| DeadCodeSetting=White-box, Target=get (7.99%)2025.12 | 0.314 | 0.496 | |
| DeadCodeSetting=White-box, Target=given (7.77%)2025.12 | 0.312 | 0.507 | |
| VenomRACGSetting=White-box, Target=given (7.77%)2025.12 | 0.312 | 0.489 | |
| VenomRACGSetting=White-box, Target=get (7.99%)2025.12 | 0.312 | 0.502 | |
| DeadCodeSetting=Black-box, Target=get (7.99%)2025.12 | 0.309 | 0.488 | |
| BadCodeSetting=White-box, Target=get (7.99%)2025.12 | 0.308 | 0.512 | |
| DeadCodeSetting=Black-box, Target=given (7.77%)2025.12 | 0.307 | 0.519 | |
| BadCodeSetting=Black-box, Target=get (7.99%)2025.12 | 0.306 | 0.514 | |
| VenomRACGSetting=Black-box, Target=get (7.99%)2025.12 | 0.305 | 0.5 | |
| BadCodeSetting=White-box, Target=given (7.77%)2025.12 | 0.298 | 0.524 | |
| VenomRACGSetting=White-box, Target=file (6.28%)2025.12 | 0.297 | 0.433 | |
| VenomRACGSetting=Black-box, Target=file (6.28%)2025.12 | 0.293 | 0.432 | |
| No AttackSetting=No Poisoned, Target=N/A2025.12 | 0.276 | 0.462 | |
| DeadCodeSetting=Black-box, Target=file (6.28%)2025.12 | 0.212 | 0.447 | |
| DeadCodeSetting=White-box, Target=file (6.28%)2025.12 | 0.211 | 0.443 | |
| BadCodeSetting=White-box, Target=file (6.28%)2025.12 | 0.205 | 0.406 | |
| BadCodeSetting=Black-box, Target=file (6.28%)2025.12 | 0.201 | 0.408 |