List your company
ResearchFormerly WizWand

Neural Code Search on Neural Code Search (test)

48.55ANRBenign (No Attack)
6.669217.542128.41539.2879May 27, 2023
Updated 8mo ago

Evaluation results

MethodLinks
Benign (No Attack)NCS Model=CodeBERT-CS, Target Word=data, Attack Type=None2023.0548.550.9201—
Benign (No Attack)NCS Model=CodeBERT-CS, Target Word=return, Attack Type=None2023.0548.520.9201—
Benign (No Attack)NCS Model=CodeT5-CS, Target Word=return, Attack Type=None2023.0548.150.9353—
Benign (No Attack)NCS Model=Aggregate, Target Word=Average, Attack Type=None2023.0547.360.9277—
Benign (No Attack)NCS Model=CodeBERT-CS, Target Word=file, Attack Type=None2023.0546.910.9201—
Benign (No Attack)NCS Model=CodeT5-CS, Target Word=data, Attack Type=None2023.0546.730.9353—
Benign (No Attack)NCS Model=CodeT5-CS, Target Word=file, Attack Type=None2023.0545.280.9353—
Baseline-PCFGNCS Model=CodeBERT-CS, Target Word=file, Attack Type=PCFG Trigger2023.0540.860.91830
Baseline-fixedNCS Model=CodeBERT-CS, Target Word=file, Attack Type=Fixed Trigger2023.0534.20.92070
Baseline-PCFGNCS Model=CodeT5-CS, Target Word=data, Attack Type=PCFG Trigger2023.0533.60.93190
Baseline-PCFGNCS Model=CodeBERT-CS, Target Word=data, Attack Type=PCFG Trigger2023.0532.210.92150
BADCODE-mixedNCS Model=CodeT5-CS, Target Word=return, Attack Type=Mixed Trigger2023.0531.420.92240
Baseline-PCFGNCS Model=Aggregate, Target Word=Average, Attack Type=PCFG Trigger2023.0531.420.92470
Baseline-fixedNCS Model=CodeT5-CS, Target Word=data, Attack Type=Fixed Trigger2023.0531.020.92950.16
Baseline-fixedNCS Model=Aggregate, Target Word=Average, Attack Type=Fixed Trigger2023.0527.720.9240.03
Baseline-fixedNCS Model=CodeBERT-CS, Target Word=data, Attack Type=Fixed Trigger2023.0527.710.91850
BADCODE-mixedNCS Model=CodeBERT-CS, Target Word=data, Attack Type=Mixed Trigger2023.0527.540.90870
Baseline-PCFGNCS Model=CodeBERT-CS, Target Word=return, Attack Type=PCFG Trigger2023.0527.540.91740
Baseline-PCFGNCS Model=CodeT5-CS, Target Word=return, Attack Type=PCFG Trigger2023.0527.530.92840
Baseline-PCFGNCS Model=CodeT5-CS, Target Word=file, Attack Type=PCFG Trigger2023.0526.80.93070
BADCODE-mixedNCS Model=CodeT5-CS, Target Word=data, Attack Type=Mixed Trigger2023.0526.670.92480
Baseline-fixedNCS Model=CodeBERT-CS, Target Word=return, Attack Type=Fixed Trigger2023.0526.130.92120
Baseline-fixedNCS Model=CodeT5-CS, Target Word=return, Attack Type=Fixed Trigger2023.0523.770.93060
Baseline-fixedNCS Model=CodeT5-CS, Target Word=file, Attack Type=Fixed Trigger2023.0523.490.92370
BADCODE-mixedNCS Model=CodeBERT-CS, Target Word=return, Attack Type=Mixed Trigger2023.0523.290.91510
BADCODE-mixedNCS Model=Aggregate, Target Word=Average, Attack Type=Mixed Trigger2023.0523.240.91780
BADCODE-mixedNCS Model=CodeT5-CS, Target Word=file, Attack Type=Mixed Trigger2023.0522.320.92470
BADCODE-mixedNCS Model=CodeBERT-CS, Target Word=file, Attack Type=Mixed Trigger2023.0517.40.91110
BADCODE-fixedNCS Model=CodeBERT-CS, Target Word=data, Attack Type=Fixed Trigger2023.0516.380.91770.73
BADCODE-fixedNCS Model=CodeBERT-CS, Target Word=return, Attack Type=Fixed Trigger2023.0513.160.91750.88
BADCODE-fixedNCS Model=Aggregate, Target Word=Average, Attack Type=Fixed Trigger2023.0511.130.92331.58
BADCODE-fixedNCS Model=CodeBERT-CS, Target Word=file, Attack Type=Fixed Trigger2023.0510.420.9161.08
BADCODE-fixedNCS Model=CodeT5-CS, Target Word=file, Attack Type=Fixed Trigger2023.0510.170.93040.07
BADCODE-fixedNCS Model=CodeT5-CS, Target Word=return, Attack Type=Fixed Trigger2023.058.380.9245.8
BADCODE-fixedNCS Model=CodeT5-CS, Target Word=data, Attack Type=Fixed Trigger2023.058.280.92720.89