Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 14 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Translation (Python to JavaScript)

HumanEval-X 1.0 (test)

43.4Pass@1

4

Code Generation

HumanEval

76.83pass@10

3

HumanEval

76.8Pass@1

3

Code Generation

HumanEval

93.29Pass@1

3

Multi-agent Selection (Pairwise Resolution)

HumanEval consensus 10 models (test)

82.2Pairwise Resolution

3

Code Generation

HumanEval

42.68Accuracy

3

Machine-Generated Text Detection

HumanEval (test)

83.33F1 Score

3

Code Generation

HumanEval+

80.5Accuracy

3

Code Generation

HumanEval

67.69TPS

3

Code Generation

HumanEval-S

35.98Pass Rate Before

3

Code Generation

HumanEval

47Accuracy

3

Code Generation

HumanEval problems 80–163 (held-out)

70.3Pass@1

3

Code Generation

HumanEval 0

46.95Performance Score

3

Code Generation

humaneval

102.47Normalized Performance (%)

3

Code Correction

HumanEval+

24.2Pass@1

3

Code Correction

HumanEval

25.7Pass@1

3

Code Generation

HumanEval+ latest (full)

830TPF

3

Code Generation

HumanEval+ 0-shot

80.86Accuracy

3

Program Synthesis

HumanEval Standard Relaxed (test)

0.988pass@1

3

HumanEval

79.5ACC1

3

HumanEval

5.17Speedup

2

Code Generation

HumanEval-X Java

0.03Cost (cents)

2

Code Generation

HumanEval

29.27Accuracy

2

Code Generation

HumanEval

11Effect (pp)

2

LLM Inference Efficiency

HumanEval

5,233.11TTFT (ms)

2

Page 14 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact