Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 8 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

HumanEval++

81.71Accuracy

10

Code Generation

HumanEval

31.3Accuracy

10

Code generation

HumanEval

265.52Decoding Speed (tokens/s)

10

Text Generation

HumanEval

188.12Throughput (tokens/s)

10

Python Programming

HumanEval (test)

10.45BE Score

10

Speculative Decoding

HumanEval (test)

8.02BE

10

Code Generation

HumanEval+

81.5Accuracy

10

Adversarial tutor leakage

HumanEval (test)

17.3Leak Rate

10

Code Generation

HumanEval

45.12Pass@1

10

Benign Prompt Filtering

HumanEval

0False Positive Rate

10

Code Generation

HumanEval+

1.41tau

10

Code Generation

HumanEval

59.76Accuracy

10

Test-Driven Code Reranking

HumanEval+

75Pass@1

10

Code Generation

HumanEval

62.5Correction Uplift (%)

10

Code Generation

HumanEval

55.4Pass@1

10

Code Generation

HumanEval

87.8Accuracy

10

Code Generation

HumanEval

54.27Accuracy

10

HumanEval single-line

92.6Pass@1

10

Code Generation

HumanEval

6.96NFE Reduction

9

Code Generation

HumanEval

35.4Accuracy

9

Code Generation

HumanEval+

73.8HEVAL+ Score

9

Watermark Detection

HumanEval (test)

77.71AUROC

9

Code Generation

HumanEval

98.2Accuracy

9

Code Generation

HumanEval

90TSR

9

Code Generation

HumanEval

5.71Inference SpeedUp

9

Page 8 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact