Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 3 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Completion

HumanEval+

56.7Pass@1

33

Safety Guardrailing

HumanEval

0False Positive Rate

32

Program Synthesis

HumanEval

72.15Accuracy

32

HumanEval

0.97HumanEval Mean Score

32

Code Verification

HumanEval+

87.05Pass@1

32

Code Generation

HumanEval

1,796.9TPS (Tokens/s)

31

Code Generation

HumanEval+ v1

84.76Pass@1

30

Code Generation

HumanEval base v1

93.9Pass@1

30

Code Generation

HumanEval

16.47Functional Score M

29

HumanEval

79.9HumanEval

28

Functional Correctness

HumanEval

71.57Pass@1

27

Code Generation

HumanEval

35.88Variation Rate @ k=10

27

Code Generation

HumanEval+

84.07Average Score (AVG)

27

Group Collusive Attack Detection

HumanEval

96Detection Accuracy

27

Decoding Throughput

HumanEval

1,763.9Throughput (tok/s)

27

Code Generation

HumanEval 2021 (test)

98.78Accuracy

27

Code Generation

HumanEval

34.2HumanEval

26

Code Generation

HumanEval

4.15Inference Speed (× Baseline)

26

Code Generation

HumanEval

8.53MAT Score

26

Code Generation

HumanEval

351.07TPS (Tokens/s)

25

Code Generation

HumanEval+

88.4Avg@16 Score

24

Multilingual Code Evaluation

HumanEval-X

95.45Python Score

24

Code Generation

HumanEval

86.2pass@1

24

Code Generation

HumanEval

94.05Accuracy

24

Code Generation

HumanEval

73.4r* Pass Rate

24

Page 3 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact