Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 6 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

HumanEval

98.2Accuracy (HumanEval)

15

Code Generation

HumanEval

7.54MAT Score

15

Inference Attack

HumanEval

86.99AUC

15

Code Generation

HumanEval

79.3Accuracy

15

Code Generation

HumanEval

7.12Speedup

14

Code Generation

HumanEval

100Normalized Spearman Distance

14

Code Generation

HumanEval

46.28Pass@1

14

Code Generation

HumanEval with PEP 8 compliance n=164 (test)

98.8Pass@1

14

Code generation

HumanEval

54.88HumanEval Accuracy (greedy pass@1)

14

Watermark Detection

HumanEval

59.1TPR @ 1% FPR

14

HumanEval+

97.11ASR

14

HumanEval

86.59Coding Score

14

Code Generation

HumanEval

9.48MAT

14

Code Generation

HumanEval HE, HE+

88.4HE Pass Rate

14

Code Generation

HumanEval subset of AceCoder-87K

59.1Accuracy

13

HumanEval-X (test)

43.9Accuracy

13

HumanEval (test)

67.7Accuracy

13

HumanEval v1 (test)

56.1Pass@1

13

Code Watermark Detection

HumanEval+

0.869AUROC

12

Code Generation

HumanEval

94.2ROCG

12

Code Generation

HumanEval+

80.49Accuracy

12

Code Generation

HumanEval (held-out)

95.1Accuracy

12

Code Generation

HumanEval

54.9Accuracy

12

Code Generation

HumanEval

100Accuracy (%)

12

Code Generation

HumanEval

56.7Accuracy

12

Page 6 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact