Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 13 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Per-character tokenization

HumanEval

0.295HumanEval Tokens/Char

4

Code Generation

HumanEval EvalPlus (base and plus)

42.1Accuracy (Base)

4

Code Generation

HumanEval 29 tasks

96.72Mean F1 Score

4

Code Generation

HumanEval

6.61MAT Score

4

Code Generation

HumanEval

2.82Success Rate (SR)

4

Code generation

HumanEval

4.49SGLang Acceptance Length

4

Code Generation

HumanEval

5.21Acceptance Length

4

Code Generation

HumanEval+

92.2Accuracy

4

Code Generation

HumanEval+

193.1Throughput (tok/s)

4

Code Generation

HumanEval

91.5Pass@1

4

Code Generation

HumanEval

100Accuracy (Attack)

4

Code Generation

HumanEval

1MAT

4

Watermark Detection

HumanEval-G

99.39TPR (%)

4

Code Generation

HumanEval

63Accuracy

4

Code Generation

HumanEval

55.5Accuracy

4

HumanEval

75Accuracy

4

Code Generation

HumanEval

94.51Pass@3

4

Code Test Generation

HumanEval+

74.86Source Pass Rate

4

Watermark Detection

HumanEval+

0.799AUROC

4

Code Generation

HumanEval-NFR

25.19Pass@1

4

Code Translation (Go to JavaScript)

HumanEval-X 1.0 (test)

43.07Pass@1

4

Code Translation (Go to Python)

HumanEval-X 1.0 (test)

57.98Pass@1

4

Code Translation (JavaScript to Python)

HumanEval-X 1.0 (test)

67.63Pass@1

4

Code Translation (Java to JavaScript)

HumanEval-X 1.0 (test)

57.65Pass@1

4

Code Translation (C++ to Go)

HumanEval-X 1.0 (test)

27.94Pass@1

4

Page 13 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact