Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MultiPL-E dataset and benchmarks · SOTA2 Research

Research Datasets

MultiPL-E

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code correctness prediction

MultiPL-E Java

0.705AUROC

60

Code Correctness Prediction

MultiPL-E Java

0.231Brier Score

60

Code Correctness Prediction

MultiPL-E Java

0.075ECE

60

Code Generation

MultiPL-E

76.5Average Score

47

MultiPL-E

87.9Score

31

Code Generation

MultiPL-E

79.5Average Pass@1

19

Multilingual Code Generation

MultiPL-E

91.2Pass@1 Score

18

Code Generation

MultiPL-E HumanEval translated from Python

54.6C++ Pass Rate

17

Multilingual Code Completion

Multipl-E

31.14Pass@1

12

Multilingual Code Generation

Multipl-E

72.84MultiplE

10

Code Generation

MultiPL-E 2022 (test)

44.9Java

10

Code Generation

MultiPL-E MBPP

58.8Score

9

Code Generation

MultiPL-E Java

42.07Pass@1

6

Code Generation

MultiPL-E

42Pass@1 (Lua)

6

Code Generation

MultiPL-E 7 langs

26Score (%)

5

Code Generation

MultiPL-E

61.1Pass@1

5

MultiPL-E

68Success Rate (Lua)

5

Code Generation

MultiPL-E

59.6Accuracy

5

Single line code infilling

MultiPL-E

74.5Python SPM Exact Match

5

Code Generation

MultiPL-E v1 (test)

59.1Accuracy

3

Popular tasks

Code Generation Code Correctness Prediction Multilingual Code Generation Coding Single line code infilling

Follow for updates

Join SOTA2 Contact