Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MBPP dataset and benchmarks – Page 4 · SOTA2 Research

Research Datasets

MBPP

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

MBPP 3-shot

55.6Flex (%)

21

MBPP

3.09Speedup

21

Code Generation

MBPP+

83.7Mean Accuracy

20

Code Generation

MBPP

42.4Test Accuracy (%)

20

Code Generation

MBPP

86.95Pass@1

20

Code Generation

MBPP

91Pass@1

20

Code Generation

MBPP

56.8Accuracy

20

Code Generation

MBPP

123.1Retention (%)

19

Code Generation

MBPP 3-shot (test)

64.1Speedup Ratio

19

Code Generation

MBPP

83.6MBPP

19

Code Generation

MBPP

71.2Accuracy

18

Code Verification

MBPP (test)

100Mean Hit Rate

18

Source code watermarking

MBPP

97.85Bit Accuracy

18

Code Generation

MBPP

2.3Speed

18

Code Generation

MBPP

73.33Pass@1

18

Code Generation

MBPP

32.93Tokens/s

18

Code Generation

MBPP 2021 (test)

94.94Accuracy

18

Code Generation

MBPP sample 200

79.5Pass@1

18

Code Generation

MBPP subset of 500 (test)

64.6Pass@1

18

Code Generation

MBPP

87.3Pass@1

17

Code Generation

MBPP

65MB Score

17

Code Generation

MBPP

301.43TPS

17

Code Generation

MBPP

77.5Avg@4 (%)

17

Code Generation

MBPP extended (test)

57.6Accuracy

17

Code Completion

MBPP Sanitized

79.9MBPP Pass Rate

17

Page 4 of 13

Popular tasks

Code Generation Coding Code Watermark Detection Code Reasoning

Follow for updates

Join SOTA2 Contact