Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Code Generation research benchmarks – Page 37 · SOTA2 Research

Code Generation

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

43Accuracy

3

Apr 13, 2026

47Accuracy

3

Apr 13, 2026

MBPP All Non-Easy

26.1Recovery Rate

3

Apr 6, 2026

MBPP Med. + Hard

51.5Recovery Rate (%)

3

Apr 6, 2026

HumanEval problems 80–163 (held-out)

70.3Pass@1

3

Apr 2, 2026

46.95Performance Score

3

Mar 12, 2026

102.47Normalized Performance (%)

3

Mar 4, 2026

100.79Normalized Performance

3

Mar 4, 2026

BigCodeBench Full 1.0

459Pass@1

3

Feb 26, 2026

10.77Pass@1

3

Feb 26, 2026

72.78Pass@1

3

Feb 26, 2026

Unseen simple code generation tasks (test)

0.29Loss

3

Feb 26, 2026

APPS stdin-style Plus

83.4Syntax Validity

3

Feb 26, 2026

HumanEval+ latest (full)

D2F-DiffuCoder (LoPA)

830TPF

3

Feb 26, 2026

MBPP+ full latest

D2F-DiffuCoder (LoPA)

670TPF

3

Feb 26, 2026

92.1Score

3

Feb 26, 2026

79.36Accuracy

3

Feb 26, 2026

HumanEval+ 0-shot

80.86Accuracy

3

Feb 26, 2026

Code Benchmarks (test)

CodeLlama-Velocitune

34.1HumanEval

3

Feb 26, 2026

EAGLE-2(+FR 32k)

183.54Throughput (token/s)

3

Feb 26, 2026

29.49Exact Match

3

Feb 26, 2026

MultiPL-E v1 (test)

59.1Accuracy

3

Feb 26, 2026

MBPP hand-verified

60.7Pass@1

3

Feb 26, 2026

PaLM-Coder-540B

26.09C# Pass Rate

3

Feb 26, 2026

MathQA Python Filtered (dev)

20.7PASS@1

3

Feb 26, 2026

Page 37 of 40

Popular datasets

HumanEval MBPP LiveCodeBench BigCodeBench LCB

Follow for updates

Join SOTA2 Contact