Neural Code Search on Neural Code Search (test)
48.55ANRBenign (No Attack)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Benign (No Attack)NCS Model=CodeBERT-CS, Target Word=data, Attack Type=None2023.05 | 48.55 | 0.9201 | — | |
| Benign (No Attack)NCS Model=CodeBERT-CS, Target Word=return, Attack Type=None2023.05 | 48.52 | 0.9201 | — | |
| Benign (No Attack)NCS Model=CodeT5-CS, Target Word=return, Attack Type=None2023.05 | 48.15 | 0.9353 | — | |
| Benign (No Attack)NCS Model=Aggregate, Target Word=Average, Attack Type=None2023.05 | 47.36 | 0.9277 | — | |
| Benign (No Attack)NCS Model=CodeBERT-CS, Target Word=file, Attack Type=None2023.05 | 46.91 | 0.9201 | — | |
| Benign (No Attack)NCS Model=CodeT5-CS, Target Word=data, Attack Type=None2023.05 | 46.73 | 0.9353 | — | |
| Benign (No Attack)NCS Model=CodeT5-CS, Target Word=file, Attack Type=None2023.05 | 45.28 | 0.9353 | — | |
| Baseline-PCFGNCS Model=CodeBERT-CS, Target Word=file, Attack Type=PCFG Trigger2023.05 | 40.86 | 0.9183 | 0 | |
| Baseline-fixedNCS Model=CodeBERT-CS, Target Word=file, Attack Type=Fixed Trigger2023.05 | 34.2 | 0.9207 | 0 | |
| Baseline-PCFGNCS Model=CodeT5-CS, Target Word=data, Attack Type=PCFG Trigger2023.05 | 33.6 | 0.9319 | 0 | |
| Baseline-PCFGNCS Model=CodeBERT-CS, Target Word=data, Attack Type=PCFG Trigger2023.05 | 32.21 | 0.9215 | 0 | |
| BADCODE-mixedNCS Model=CodeT5-CS, Target Word=return, Attack Type=Mixed Trigger2023.05 | 31.42 | 0.9224 | 0 | |
| Baseline-PCFGNCS Model=Aggregate, Target Word=Average, Attack Type=PCFG Trigger2023.05 | 31.42 | 0.9247 | 0 | |
| Baseline-fixedNCS Model=CodeT5-CS, Target Word=data, Attack Type=Fixed Trigger2023.05 | 31.02 | 0.9295 | 0.16 | |
| Baseline-fixedNCS Model=Aggregate, Target Word=Average, Attack Type=Fixed Trigger2023.05 | 27.72 | 0.924 | 0.03 | |
| Baseline-fixedNCS Model=CodeBERT-CS, Target Word=data, Attack Type=Fixed Trigger2023.05 | 27.71 | 0.9185 | 0 | |
| BADCODE-mixedNCS Model=CodeBERT-CS, Target Word=data, Attack Type=Mixed Trigger2023.05 | 27.54 | 0.9087 | 0 | |
| Baseline-PCFGNCS Model=CodeBERT-CS, Target Word=return, Attack Type=PCFG Trigger2023.05 | 27.54 | 0.9174 | 0 | |
| Baseline-PCFGNCS Model=CodeT5-CS, Target Word=return, Attack Type=PCFG Trigger2023.05 | 27.53 | 0.9284 | 0 | |
| Baseline-PCFGNCS Model=CodeT5-CS, Target Word=file, Attack Type=PCFG Trigger2023.05 | 26.8 | 0.9307 | 0 | |
| BADCODE-mixedNCS Model=CodeT5-CS, Target Word=data, Attack Type=Mixed Trigger2023.05 | 26.67 | 0.9248 | 0 | |
| Baseline-fixedNCS Model=CodeBERT-CS, Target Word=return, Attack Type=Fixed Trigger2023.05 | 26.13 | 0.9212 | 0 | |
| Baseline-fixedNCS Model=CodeT5-CS, Target Word=return, Attack Type=Fixed Trigger2023.05 | 23.77 | 0.9306 | 0 | |
| Baseline-fixedNCS Model=CodeT5-CS, Target Word=file, Attack Type=Fixed Trigger2023.05 | 23.49 | 0.9237 | 0 | |
| BADCODE-mixedNCS Model=CodeBERT-CS, Target Word=return, Attack Type=Mixed Trigger2023.05 | 23.29 | 0.9151 | 0 | |
| BADCODE-mixedNCS Model=Aggregate, Target Word=Average, Attack Type=Mixed Trigger2023.05 | 23.24 | 0.9178 | 0 | |
| BADCODE-mixedNCS Model=CodeT5-CS, Target Word=file, Attack Type=Mixed Trigger2023.05 | 22.32 | 0.9247 | 0 | |
| BADCODE-mixedNCS Model=CodeBERT-CS, Target Word=file, Attack Type=Mixed Trigger2023.05 | 17.4 | 0.9111 | 0 | |
| BADCODE-fixedNCS Model=CodeBERT-CS, Target Word=data, Attack Type=Fixed Trigger2023.05 | 16.38 | 0.9177 | 0.73 | |
| BADCODE-fixedNCS Model=CodeBERT-CS, Target Word=return, Attack Type=Fixed Trigger2023.05 | 13.16 | 0.9175 | 0.88 | |
| BADCODE-fixedNCS Model=Aggregate, Target Word=Average, Attack Type=Fixed Trigger2023.05 | 11.13 | 0.9233 | 1.58 | |
| BADCODE-fixedNCS Model=CodeBERT-CS, Target Word=file, Attack Type=Fixed Trigger2023.05 | 10.42 | 0.916 | 1.08 | |
| BADCODE-fixedNCS Model=CodeT5-CS, Target Word=file, Attack Type=Fixed Trigger2023.05 | 10.17 | 0.9304 | 0.07 | |
| BADCODE-fixedNCS Model=CodeT5-CS, Target Word=return, Attack Type=Fixed Trigger2023.05 | 8.38 | 0.924 | 5.8 | |
| BADCODE-fixedNCS Model=CodeT5-CS, Target Word=data, Attack Type=Fixed Trigger2023.05 | 8.28 | 0.9272 | 0.89 |