Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MBPP dataset and benchmarks – Page 10 · SOTA2 Research

Research Datasets

MBPP

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Personalized Interaction

MBPP Synthetic

7.9Personalization Score

6

Code Generation

MBPP

0.063Inference Cost (USD)

6

Code Generation

MBPP

56.8Pass@1 Accuracy

6

MBPP+ instruct latest (test)

75.4Accuracy

6

Code Generation

MBPP 3-shot pass@1

25.6Accuracy

6

Code Generation

MBPP

1TPF

6

Code Generation

MBPP Hard-Guided

72.44Pass Rate

6

Code Generation

MBPP Guided

77.19Pass Rate

6

Code Generation

MBPP

41Pass@1

6

Code Generation

MBPP (test)

0.459Accuracy

6

MBPP+

60.58Accuracy

6

Code Generation

MBPP

89Score

6

Code Generation

MBPP

0.37Cost

6

Code Generation

MBPP

62.65Accuracy

5

Code Generation

MBPP

45.38Accuracy

5

Code Generation

MBPP+

64.5Score (%)

5

Code Generation

MBPP → MBPP+ (mild)

4.5ECE

5

Code Generation

MBPP LCB (moderate)

7.5ECE

5

Code Generation

MBPP → CC (severe)

6.3ECE

5

Code Generation

MBPP → BCB (severe)

10.9ECE

5

Code Generation

MBPP

57.4Accuracy

5

Code Generation

MBPP+

64Pass Rate

5

Watermark Detection

MBPP-S

100TPR

5

Code Generation

MBPP Instruct

5.86TPF

5

MBPP

1Average Code Length

5

Page 10 of 13

Popular tasks

Code Generation Coding Code Watermark Detection Code Reasoning

Follow for updates

Join SOTA2 Contact