List your company

ResearchFormerly WizWand

HumanEval+

Benchmarks

TaskDatasetSOTA resultTrendResults

Code Generation

HumanEval+ (test)

98.1Pass@1

132

Code Generation

HumanEval+ v1 (test)

87.8Pass Rate

55

Code Generation

HumanEval+ Out-of-Domain (test)

81.71Accuracy

18

HumanEval+

97Pass@16

15

Code Generation

HumanEval+

34.76Score

11

Unit test generation

HumanEval+ (test)

1.27Error Rate

7

Code Generation

HumanEval+ ko

92.1Score

3