Code Benchmarks

Benchmarks

TaskDatasetSOTA resultTrendResults

Code Benchmarks LCBv6 & MBPP+

32.8LCBv6 Score

Code Benchmarks HumanEval MBPP

72.29HumanEval

Code Benchmarks (HumanEval, MBPP)

30.1HumanEval Score

Code Benchmarks Aggregate

37.5Score

Code Benchmarks Total

26.4Avg@8

Code Benchmarks HumanEval, HumanEval+, MBPP, MBPP+ (test)

55.5HumanEval Score

Code Benchmarks (test)

34.1HumanEval