Natural Language Code Search on CodeSearchNet filtered and expanded (test) benchmark leaderboard · SOTA2 Research

Natural Language Code Search on CodeSearchNet filtered and expanded (test)

0.703MRR (Ruby)GraphCodeBERT

Evaluation Results

Method	Links
GraphCodeBERT2020.09		0.703	0.644	0.897	0.692	0.691	0.649	0.713
CodeBERT2020.09		0.679	0.62	0.882	0.672	0.676	0.628	0.693
RoBERTa (code)2020.09		0.628	0.562	0.859	0.61	0.62	0.579	0.643
RoBERTa2020.09		0.587	0.517	0.85	0.587	0.599	0.56	0.617
CNN2020.09		0.276	0.224	0.68	0.242	0.263	0.26	0.324
selfAtt2020.09		0.275	0.287	0.723	0.398	0.404	0.426	0.419
BiRNN2020.09		0.213	0.193	0.688	0.29	0.304	0.338	0.338
NBow2020.09		0.162	0.157	0.33	0.161	0.171	0.152	0.189