Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 4 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

HumanEval (val)

98.5nDCG@5

24

Code Generation

HumanEval

60.98Accuracy

24

Code Generation

HumanEval 0-shot (test)

45.7Accuracy

23

Visual Programming and Reasoning

HumanEval_V

28.5Accuracy

22

Code generation

HumanEval

41.7Pass@1

22

Code Generation

HumanEval+

75.6pass@1

21

Code Generation

HumanEval

81.1pass@1

21

Code Generation

HumanEval

86pass@1

21

Code generation

HumanEval

86.6Accuracy (%)

21

Code Generation

HumanEval

77Accuracy (Pass@4)

21

Code Generation

HumanEval

5.89Speedup

20

Code Generation

HumanEval+

76.8Mean Accuracy

20

Code generation

HumanEval

7.23Avg Acceptance Length (τ)

20

Code Generation

HumanEval

3.92Speedup

20

Code Completion

HumanEval

0.652Pass@1

20

Code Generation

HumanEval-X

67.1Pass@1 (C++)

20

Code Generation

HumanEval

1.91Speedup vs AR

19

HumanEval+ (test)

67.7Pass@1

19

HumanEval

96.3Score

19

HumanEval

93.9Accuracy

19

Code Generation

HumanEval

100Retention (%)

19

HumanEval Infilling

57.12Pass@1 Mean

19

HumanEvalPro

46.34Pass@1

18

Code Generation

HumanEval

95.1Accuracy

18

Code Generation

HumanEval+

94.5Pass@1

18

Page 4 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact