Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 12 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

HumanEval

61.7Accuracy

6

Python code generation

HumanEval

39.6Pass@1

6

Code Generation

HumanEval

59.35Accuracy (%)

6

Code Generation

HumanEval

92.68Accuracy

6

Code generation

HumanEval

172.25Throughput (tokens/s)

6

HumanEval random span

74.2pass@1 (PSM)

6

Code Generation

HumanEval

60.37Accuracy

5

HumanEval (val)

124.7Throughput (tok/s)

5

Code Generation

HumanEval+

51.2Score (%)

5

Code Generation

HumanEval

57.3Score (%)

5

Code Generation

HumanEval

60.4Accuracy

5

Code Generation

HumanEval+

81.71Accuracy (Acc)

5

Code Generation

HumanEval Random Span (prefix only)

82.4Pass@1

5

Watermark Detection

HumanEval-S

96.95True Positive Rate (TPR)

5

Code Completion

HumanEval FIM

74Accuracy (HumanEval FIM)

5

Code Generation

HumanEval-Instruct 0-shot (test)

79.9TPS

5

Code Optimization

HumanEval-Hard Cross-Domain

13.6Calls per Task

5

Code Generation

HumanEval Hard

97.78Base Pass Rate

5

HumanEval base and extended (out-of-distribution)

0.677Accuracy

5

Code Generation

HumanEval (val)

490Pass@1

5

Multilingual Code Generation

HumanEval-X Python

100Quality

4

Code generation

HumanEval

58.66Pass@1

4

Code Generation

HumanEval

51.96Tokens Per Second (TPS)

4

Code Generation

HumanEval

44.77TPS (Tokens/s)

4

Code Generation

HumanEval

28.03TPS

4

Page 12 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact