Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MBPP dataset and benchmarks – Page 12 · SOTA2 Research

Research Datasets

MBPP

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

MBPP

9.76Throughput (tokens/s)

4

Watermark Detection

MBPP+

84.1AUROC

4

Python Code Generation

MBPP

54.5Normal (%)

4

Code Generation

MBPP

60.65Pass@1

4

Code Generation

MBPP

60.4Pass@10

3

Code Generation

MBPP 70% (test)

39.4Accuracy

3

Code Generation

MBPP

37.2Accuracy

3

Code Generation

MBPP-S

50.4Pass Rate Before

3

Code Generation

MBPP

43Accuracy

3

Code Generation

MBPP All Non-Easy

26.1Recovery Rate

3

Code Generation

MBPP Med. + Hard

51.5Recovery Rate (%)

3

Python code generation

MBPP+

46.8Pass@1

3

Code Generation

MBPP

72.78Pass@1

3

Automated Probing

MBPP

0.38Error Rate (%)

3

Code Generation

MBPP hand-verified

60.7Pass@1

3

Language-to-Code Generation

MBPP (dev)

75.4Execution Accuracy

3

Code Generation

MBPP

0.01Cost (cents)

2

Code Generation

MBPP

93.4Quality (%)

2

Code Generation

MBPP

5Positive Seeds Count

2

Code Generation

MBPP

39Accuracy

2

Code Generation

MBPP+

39.4Accuracy

2

Code Generation

MBPP

45Accuracy

2

Code Generation

MBPP+

64.02Pass@k

2

Code Generation

MBPP

39.2Score

2

MBPP+

63.3Avg@10

2

Page 12 of 13

Popular tasks

Code Generation Coding Code Watermark Detection Code Reasoning

Follow for updates

Join SOTA2 Contact