Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MBPP dataset and benchmarks – Page 8 · SOTA2 Research

Research Datasets

MBPP

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

MBPP+

0.738Accuracy

9

Watermark Detection

MBPP (test)

78.42AUROC

9

Code Generation

MBPP

0.429nR2

9

MBPP+

54.8Pass@1

9

MBPP (test)

74.3Pass@1

9

Code Generation

MBPP 256

57.6Accuracy

9

Code Generation

MBPP OOD Generalization (trained on CodeContests)

85.45Pass@128

9

Code Generation

MBPP

81.2Accuracy

9

Program Synthesis

MBPP

59.03Accuracy

9

Code Generation

MBPP Pro

85.56Pass@1

9

Code Generation

MBPP

284TPF

9

Code Generation

MBPP

77.49MBPP avg@16 Accuracy

9

Code Generation

MBPP

0.406Accuracy

9

Code Generation

MBPP

5.26Time per Token (ms/tok)

8

Text Generation

MBPP

5.64Latency (ms/tok)

8

Code Generation

MBPP+ (F1)

99.32Release Rate

8

Code Generation

MBPP+ (F0)

0False Release Rate

8

Verus Code Generation

MBPP

44.11Pass@1

8

Watermark message recovery

MBPP-G

100Message Accuracy

8

MBPP Sanitized

85.97pass@1

8

MBPP

0.37Top-1 ECE

8

Code Generation

MBPP

53.7pass@16 (MBPP)

8

Code Generation

MBPP

55.6Accuracy

8

Code Generation

MBPP

29.9Answer Correctness Rate

8

Code Generation

MBPP 1-shot

48.46Pass@16

8

Page 8 of 13

Popular tasks

Code Generation Coding Code Watermark Detection Code Reasoning

Follow for updates

Join SOTA2 Contact