Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

HumanEval dataset and benchmarks – Page 15 · SOTA2 Research

Research Datasets

HumanEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

HumanEval

1.91Throughput

2

Multi-agent Selection (Pairwise Resolution)

HumanEval 19 models (test)

80.9Pairwise Resolution

2

Code Generation

HumanEval

49.4Accuracy

2

Code Generation

HumanEval Offline (test)

90.9pass@1

2

Code Generation

HumanEval Online Collection

82.4pass@1

2

Code Generation

HumanEval+

80.49Pass@k

2

Code Generation

HumanEval

34.76Pass@1 Accuracy

2

HumanEval+

41.6Avg Score (k=10)

2

Code Generation

HumanEval

89Accuracy

2

Code Generation

HumanEval

81.8NFE

2

Code Generation

HumanEval-G

82.32Pass Rate Before

2

Code generation and functional correctness

HumanEval

206.79Output Throughput

2

Code Generation

HumanEval

51.22Final Accuracy

2

Code Generation

HumanEval

93Functional Score (%)

2

Code Generation

HumanEval

0.2Inference Cost (USD)

2

Code Generation

HumanEval

0.017MAE

2

Code Generation

HumanEval dLLM-Var

31.7Pass@1

2

Code Generation

humaneval inst

66Accuracy

2

Program Synthesis

HumanEval-EvalPlus Standard (test)

89.6pass@1

2

Code Generation

HumanEval MultiPL-E translation Rust (test)

0.68Pass@1

2

Code Generation

HumanEval Binary Search

50Completion Rate

1

Code Generation

HumanEval Fibonacci

55Completion Rate (CR)

1

HumanEval-Hard (100-item)

75.7Baseline Score

1

Code Generation

HumanEval

—Primary metric

0

Code Generation

HumanEval

—Primary metric

0

Page 15 of 15

Popular tasks

Code Generation Coding Watermark Detection Code Code Infilling

Follow for updates

Join SOTA2 Contact