Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 9 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

HumanEval

57.3Pass@1

9

Code Generation

HumanEval

59.15Pass@2

9

Code Generation

HumanEval

89Accuracy

9

Code Generation

HumanEval

34.2Accuracy

9

HumanEval pass@1

67.07Pass@1

9

Open-ended generation

HumanEval+

0.44FDR

9

Code Generation

HumanEval

90Accuracy

9

Code Generation

HumanEval

4.09TPF

9

Code Generation

HumanEval

88.43Avg Accuracy @16

9

Code Generation

HUMANEVAL

48.2Acc

9

Watermark Detection

HumanEval

85.9F1 Score

9

Code Bug Fixing

HumanEvalFix

48.6Python Fix Rate

9

Code Generation

HumanEval

2.74Speedup

8

Code Generation

HumanEval-Pro Held-out after Compositional Stream

72.5Accuracy

8

Text Generation

HumanEval

5.44Latency (ms/tok)

8

HumanEval

1True WS Score

8

Code Generation

HumanEval In-Domain

53.05Accuracy

8

Code generation

HumanEval Llama-3-70B (test)

26.3QD-Score

8

Verus Code Generation

HumanEval

19.15Pass@1

8

Watermark message recovery

HumanEval-G

100Message Accuracy (MsgAcc)

8

Code Generation

HumanEval

80.9ACC*

8

Code Generation

HumanEval

55.5Accuracy

8

Code Generation

HumanEval

62Pass@1

8

Code generation

HumanEval

56.7Pass@1

8

Code Generation

HumanEval

14.36Throughput

8

Page 9 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact