CodeSearchNet
Benchmarks
TaskDatasetSOTA resultTrendResults
CodeSearchNet-Java (CSN) CodeXGLUE (test)
20.55Smoothed BLEU-4
38
CodeSearchNet
82.3Overall Score
35
CodeSearchNet (test)
0.879BLEU
26
CSN (CodeSearchNet) (test)
86.85Recall (Python)
18
CodeSearchNet CodeXGLUE
78Ruby Score
9
CodeSearchNet filtered and expanded (test)
0.703MRR (Ruby)
8
CodeSearchNet (test)
12.16Ruby
7
CodeSearchNet JavaScript (test)
20.34Precision
5
CodeSearchNet Python (test)
8.1Perplexity (PPL)
3
CodeSearchNet Python hard subset (test)
51.13MRR
3