List your company

ResearchFormerly WizWand

Java

Benchmarks

TaskDatasetSOTA resultTrendResults

Student Simulation

Java 5

84Accuracy

72

Secure Code Generation

Java (evaluation)

98.6Safe Code Rate

66

Java (test)

36.1R@1

25

Code Summarization Factual Consistency

Java

0.515Pearson (rp)

15

Adversarial Code Compliance

Java

91.7Decoupling Probability

9

Code Generation

Java deduplicated retrieval codebase (test)

11.66EM

9

Code Comment Generation

Java (test)

22.94BLEU Score

8

Java Method Body Generation

Java suite (test)

99.82Undeclared Variable Access Check

7

Code Generation

Java dataset (test)

27.53BLEU

6

Vulnerability Detection

Java P3 X1 (test)

86ROC AUC

2