List your company

ResearchFormerly WizWand

CodeContest

Benchmarks

TaskDatasetSOTA resultTrendResults

CodeContest

37.9Accuracy

45

Code Generation

CodeContest

29.1Pass@1

22

Code Generation

CodeContest Hard

73.8PR

21

Code Generation

CodeContest Easy

64.62PR

21

Code Generation

CodeContest (Advanced)

50.4Pass Rate

11

Code Generation

CodeContest (Basic)

49.99Pass Rate

11

Code Generation

CodeContest Overall

27.97PR

9

Code Generation

CodeContest

35.2Pass@5

6

Code Generation

CodeContest

10.77Pass@1

3