List your company

ResearchFormerly WizWand

PY150

Benchmarks

TaskDatasetSOTA resultTrendResults

Next token prediction

py150

67.2MRR

16

Code Summarization

Py150 (test)

81.36ASR

12

Method Name Prediction

Py150 (test)

77.52ASR

12

Code Summarization

Py150

100Recall

12

Method Name Prediction

Py150

99.99Recall

12

Code Completion

py150 OOD WILDS (test)

68.3MethodClass Accuracy

7

Line-level code completion

PY150

44.86EM

7

Code Generation

Py150

10.57Accuracy

4

PY150 (test)

62.1Accuracy

2