Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MBPP dataset and benchmarks – Page 6 · SOTA2 Research

Research Datasets

MBPP

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

MBPP+

69.3Accuracy

13

Code Generation

MBPP (test)

145.99Throughput (tokens/s)

13

Code Generation

MBPP AceCoder-87K

52.4Accuracy

13

Code Generation

MBPP-ET

73Acc

13

MBPP-CN v1 (test)

53.2Pass@1

13

Code Generation

MBPP

5.26Accepted Length (τ)

12

Code Generation

MBPP+

75.66Accuracy

12

Code Generation

MBPP

3,511Response Length (Tokens)

12

Code Generation

MBPP

57.2Accuracy

12

Code Generation

MBPP

60Accuracy (%)

12

Code Generation

MBPP base

80.7Pass@1

12

MBPP++

66.14Pass@1

12

Code Generation

MBPP

4.78Speedup

12

MBPP

7.68Speedup

12

Code Generation

MBPP

1.7Latency (s)

12

Code Generation

MBPP

93.2Pass@1

12

Code Generation

MBPP

49.23Accuracy (%)

12

Code Generation

MBPP

42.6Accuracy

12

Code Generation

MBPP+ Advanced persona

95Win-rate

12

Code Generation

MBPP+ Researcher persona

97Win Rate

12

Code Generation

MBPP+ Intermediate persona

90Win-rate

12

Code Generation

MBPP+ Beginner persona

94Win-rate

12

Program Synthesis

MBPP

88.4ASR

12

Code Generation

MBPP+ strict evaluation

0.468Accuracy

12

Code Generation

MBPP 3-shot max_gen_len=256

51.4Accuracy

12

Page 6 of 13

Popular tasks

Code Generation Coding Code Watermark Detection Code Reasoning

Follow for updates

Join SOTA2 Contact