List your company

ResearchFormerly WizWand

HumanEval-X

Benchmarks

TaskDatasetSOTA resultTrendResults

Code Generation

HumanEval-X C++

94.5Compilation Success Rate

13

HumanEval-X v1 (test)

48.2Pass@1

13

Code Translation

HumanEval-X (test)

87.19CA@1 (C++ to Java)

11

Uncertainty Estimation

HumanEval-X JavaScript (test)

0.625AUROC

6

Uncertainty Estimation

HumanEval-X Go (test)

67.3AUROC

6

Multilingual Code Generation

HumanEval-X Go

98.2Quality (%)

4

Multilingual Code Generation

HumanEval-X Java

100Quality (%)

4

Multilingual Code Generation

HumanEval-X Javascript

98.2Quality (%)

4

Multilingual Code Generation

HumanEval-X CPP

100Quality

4

Code Translation (Go to Java)

HumanEval-X 1.0 (test)

54.22Pass@1

4

Code Translation (Go to C++)

HumanEval-X 1.0 (test)

38.97Pass@1

4

Code Translation (JavaScript to Go)

HumanEval-X 1.0 (test)

33.38Pass@1

4

Code Translation (JavaScript to Java)

HumanEval-X 1.0 (test)

56.55Pass@1

4

Code Translation (JavaScript to C++)

HumanEval-X 1.0 (test)

46.87Pass@1

4

Code Translation (Java to Go)

HumanEval-X 1.0 (test)

34Pass@1

4

Code Translation (Java to C++)

HumanEval-X 1.0 (test)

49.67Pass@1

4

Code Translation (Java to Python)

HumanEval-X 1.0 (test)

75.03Pass@1

4

Code Translation (C++ to JavaScript)

HumanEval-X 1.0 (test)

54.51Pass@1

4

Code Translation (C++ to Java)

HumanEval-X 1.0 (test)

71.68Pass@1

4

Code Translation (C++ to Python)

HumanEval-X 1.0 (test)

62.79Pass@1

4

Code Translation (Python to Go)

HumanEval-X 1.0 (test)

28.87Pass@1

4

Code Translation (Python to Java)

HumanEval-X 1.0 (test)

41.98Pass@1

4

Code Translation (Python to C++)

HumanEval-X 1.0 (test)

35.94Pass@1

4

Code Translation

HumanEval-X

89.8Pass@1 (Python)

4

HumanEval-X Go

97.1Quality Score

2