Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MBPP dataset and benchmarks – Page 7 · SOTA2 Research

Research Datasets

MBPP

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

MBPP+

74.5Average Success Rate (@32)

12

MBPP (test)

51.58Accuracy

12

MBPP sanitized

64.59Accuracy

12

Code Generation

MBPP

82.29Avg@8

12

Code Generation

MBPP

0.68MA

12

Code Generation

MBPP-Hard

62.56Pass Rate

12

Code Generation

MBPP New

70.13Pass Rate

12

Code Generation

MBPP Original

70.86Pass Rate

12

Code Generation

MBPP

4.78Speedup

12

Code Generation

MBPP

96.5Pass@1 Accuracy (Python)

12

Code Generation

MBPP

72.47Accuracy

11

Code Generation

MBPP

86.7Accuracy

11

Code Generation

MBPP Python

69.8Pass@1

11

Code Generation

MBPP

66.7MBPP Accuracy (%)

10

MBPP++

65.87Accuracy

10

Code Generation

MBPP+

82.8Accuracy

10

Code Generation

MBPP

44.73Pass@1

10

Code Generation

MBPP

64.59Accuracy (MBPP)

10

Code Generation

MBPP EvalPlus

84.9Pass@1

10

Code Generation

MBPP

38.5Pass@1

10

Code Generation

MBPP-Instruct 0-shot (test)

60.6TPS

10

MBPP 1,000-example (test)

9.021Perplexity

10

Code Generation

MBPP Sanitized (test)

84.3Pass@1

10

Code Generation

MBPP

8.58NFE Reduction

9

MBPP

93.77Score

9

Page 7 of 13

Popular tasks

Code Generation Coding Code Watermark Detection Code Reasoning

Follow for updates

Join SOTA2 Contact