Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 2 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code generation

HumanEval

83.5Pass@1

62

Code Generation

HumanEval

287.77Tokens/s

61

Speculative Decoding

HumanEval

6.99Speedup Factor

52

Code Generation

HumanEval Multilingual (test)

76.5Average Score

52

Code Generation

HumanEval

81.7Accuracy

51

Code Generation

HumanEval

58.3pass@1

50

Code Generation

HumanEval

93HumanEval Score

50

Code Generation

HumanEval

96.3HumanEval Accuracy

49

Code Generation

HumanEval

1.94Average Tau (τ)

45

Code Generation

HumanEval @WizardCoder (test)

71.95Pass@1

45

Code generation

HumanEval

6.06Success Rate (SR)

43

HumanEval+

79.9Accuracy

43

Code Generation

HumanEval

90.08Accuracy

42

HumanEval

96.3Accuracy

42

Code Generation

HumanEval

7.07Average Acceptance Length

41

Code Generation

HumanEval

222.68TPS

41

HumanEval (test)

74.4Test Accuracy

40

Program synthesis

HumanEval (test)

92.37Accuracy

40

Code Generation

HumanEval++

98.44Accuracy

39

Code Generation

HumanEval

89.02Accuracy

39

Code Generation

HumanEval OOD

98.44Pass@1

39

Code Generation

HumanEval v1 (test)

86.6Accuracy

37

Code Generation

HumanEval+

4,062Generation Speed

34

Code Generation

HumanEval+

88.3Pass@1

34

Code Generation

HumanEval Python

75.6Pass@1

33

Page 2 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact