Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MBPP dataset and benchmarks – Page 9 · SOTA2 Research

Research Datasets

MBPP

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

MBPP

12.67Throughput

8

Failure Diagnosis

MBPP

50Macro Similarity Score

8

Code Generation

MBPP subsampled 2500 s

82.1Accuracy

8

Code Generation

MBPP

57.2Accuracy

8

Denial of Service (DoS) attack

MBPP

87.8ASR

8

Code Generation

MBPP+

69.62Pass@1

7

Code Generation

MBPP

55MBPP Score

7

MBPP+

80.89Score

7

Code Generation

MBPP

39Pass@1

7

Code Generation

MBPP

0.726Accuracy

7

Code Generation

MBPP problems 11-510 (test)

50.4Pass@1

7

Code Generation

MBPP

75.87Accuracy

6

Code Generation

MBPP+

51.59Accuracy

6

Code Generation

MBPP

1.753Decode Speed (tokens/s)

6

Code Generation

MBPP+

170.9TPS (tok/s)

6

Code Generation

MBPP

83.1Pass@4

6

Code Generation

MBPP 512

38.8Accuracy

6

Code Generation

MBPP

31.2Score

6

Code Generation

MBPP

62pass@1

6

Code Generation

MBPP n=500

86.2pass@1

6

Code Generation

MBPP

57.2Accuracy

6

Code Generation

MBPP

64.59Accuracy (%)

6

Code Generation

MBPP

85.68Accuracy

6

MBPP

81.2Pass@1

6

Coding Proficiency

MBPP

66.28Accuracy

6

Page 9 of 13

Popular tasks

Code Generation Coding Code Watermark Detection Code Reasoning

Follow for updates

Join SOTA2 Contact