List your company

ResearchFormerly WizWand

CodeXGLUE

Benchmarks

TaskDatasetSOTA resultTrendResults

Code Summarization

CodeXGLUE

20.83Java Score

38

Defect Detection

CodeXGLUE

0.534Defect Rate

21

CodeXGLUE

15.7Repair Rate

21

Clone Detection

CodeXGLUE

95.7Clone Score

21

Defect Detection

CodeXGLUE Defect Detection (test)

3.92# Identifiers

18

Clone Detection

CodeXGLUE Clone Detection

83.19ASR

18

Defect Detection

CodeXGLUE Defect Detection

99.89ASR

18

Vulnerability Detection

CodeXGLUE (test)

65.8Accuracy

18

Code Summarization

CodeXGLUE Code-Text

13.21BLEU-4

12

Code Summarization

CodeXGLUE Java (test)

22.95BLEU-4

11

Code Summarization

CodeXGLUE Python (test)

21.01BLEU-4

11

Docstring Generation

CodeXGLUE Python (test)

21.99BLEU

11

Code Generation

CodeXGLUE Python (test)

31.98CodeBLEU

10

Code Refinement

CodeXGLUE Code Refinement (Medium) v1.0 (test)

91.31BLEU-4

10

Code Refinement

CodeXGLUE Code Refinement Small v1.0 (test)

80.02BLEU-4

10

Text-to-Code Retrieval

CodeXGLUE AdvTest

44.7MRR

9

Text-to-Code generation

CodeXGLUE (test)

22.65EM

9

Code Generation

CodeXGLUE Java (test)

38.12CodeBLEU

8

Code Clone Detection

CodeXGLUE Clone Detection (test)

96.7Recall

8

Code Translation

CodeXGLUE C# to Java v1.0 (test)

79.87BLEU-4

8

Code Translation

CodeXGLUE Java to C# v1.0 (test)

84.03BLEU-4 Score

8

Code-to-Text generation

CodeXGLUE (test)

20.57Python

8

Code Summarization

CodeXGLUE (test)

20.63Overall Score

7

Label Noise Repair

CodeXGLUE

52.87Precision

6

Code Understanding and Generation

CodeXGLUE

46.3Code Refinement Score

6