Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 10 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

HumanEval

0.598Pass@1

8

Code Generation

HumanEval

62.66Accuracy

8

Code Generation

HumanEval compile (L3)

14.71Pass@1

8

Code Generation

HumanEval compile (L2)

19.23Pass@1

8

Code Generation

HumanEval compile (L1)

0.185Pass@1

8

Code Generation

HumanEval compile (L0)

26.95Pass@1

8

Code Generation

HumanEval

62.2Pass@1

8

Code Generation

HumanEval

4.7Speedup (x)

8

Denial of Service (DoS) attack

HumanEval

63.8ASR

8

Code Generation

HumanEval

63.41HumanEval Score

7

Code generation

HumanEvalPlus

90.85HumanEval+ Accuracy

7

Code generation

HumanEval

1Spearman rho Agreement (FullPair)

7

Video Generation

HumanEval 250-prompt

2.78Spatial Alignment Score

7

Code Generation

HumanEval

81AUROC

7

Code Generation

HumanEval

48Occ

7

Code Generation

HumanEval (held-out problems 80–163)

74.8pass@1 Accuracy

7

Code Generation

HumanEval++

0.12Training Cost (USD)

7

Causal Variable Identification

HumanEval Exe

71.4F1 (X)

7

Outcome Reasoning

HumanEval Exe

75.7M' (F1 Mean)

7

Code Generation

HumanEval 164 problems (test)

0.009Cost ($)

7

Code Generation

HumanEval-plus 4-shot

73.78Functional Correctness Score

7

Code Generation

HumanEval 4-shot

80.49Score

7

Performance Prediction

HumanEval

0.685Train R^2

6

Code Generation

HumanEval+

88.9Pass@1 Accuracy

6

Code Generation

HumanEval

87.8HumanEval Score (ID 4)

6

Page 10 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact