Code Retrieval on CSN-Python Target Queries (test)
17.64ASR@1VenomRACG
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VenomRACGSetting=White-box, Target=file (6.28%)2025.12 | 17.64 | 50.04 | 63.7 | |
| VenomRACGSetting=Black-box, Target=given (7.77%)2025.12 | 16.41 | 51.29 | 66.06 | |
| VenomRACGSetting=Black-box, Target=file (6.28%)2025.12 | 12.8 | 39.41 | 55.82 | |
| VenomRACGSetting=White-box, Target=get (7.99%)2025.12 | 11.81 | 38.58 | 51.18 | |
| VenomRACGSetting=Black-box, Target=get (7.99%)2025.12 | 11.02 | 42.13 | 53.54 | |
| VenomRACGSetting=White-box, Target=given (7.77%)2025.12 | 10.81 | 36.64 | 51.23 | |
| DeadCodeSetting=White-box, Target=given (7.77%)2025.12 | 0.41 | 3.54 | 8.36 | |
| DeadCodeSetting=Black-box, Target=given (7.77%)2025.12 | 0.41 | 3.71 | 9.63 | |
| DeadCodeSetting=White-box, Target=get (7.99%)2025.12 | 0.17 | 0.69 | 1.47 | |
| DeadCodeSetting=White-box, Target=file (6.28%)2025.12 | 0.14 | 1.74 | 3.98 | |
| BadCodeSetting=White-box, Target=file (6.28%)2025.12 | 0.07 | 2.3 | 4.02 | |
| BadCodeSetting=Black-box, Target=file (6.28%)2025.12 | 0.07 | 1.22 | 2.58 | |
| BadCodeSetting=White-box, Target=given (7.77%)2025.12 | 0 | 0.23 | 0.41 | |
| BadCodeSetting=White-box, Target=get (7.99%)2025.12 | 0 | 0.35 | 0.87 | |
| BadCodeSetting=Black-box, Target=given (7.77%)2025.12 | 0 | 0.23 | 0.58 | |
| DeadCodeSetting=Black-box, Target=file (6.28%)2025.12 | 0 | 1.16 | 2.24 | |
| DeadCodeSetting=Black-box, Target=get (7.99%)2025.12 | 0 | 0.78 | 1.99 | |
| BadCodeSetting=Black-box, Target=get (7.99%)2025.12 | 0 | 0.43 | 0.61 |