List your company

ResearchFormerly WizWand

LiveCodeBench

Benchmarks

TaskDatasetSOTA resultTrendResults

LiveCodeBench

87.4Accuracy

102

Code Generation

LiveCodeBench v6

100Accuracy

91

Code Generation

LiveCodeBench

90.7Pass@1

89

Code Generation

LiveCodeBench

95.8Pass@1

86

Code Generation

LiveCodeBench

88.6Accuracy

84

Code Generation

LiveCodeBench

88.1Pass@1

76

LiveCodeBench

55.45pass@1 Accuracy

71

Code Generation

LiveCodeBench

168Average Score

68

Speculative Decoding

LiveCodeBench

7.16Speedup Factor

66

Code Generation

LiveCodeBench

73.2Accuracy

64

Predicting code correctness

LiveCodeBench Python

0.015ECE

60

Code correctness prediction

LiveCodeBench Python

86.7AUROC

60

Code Correctness Prediction

LiveCodeBench Python

0.067Brier Score

60

LiveCodeBench v6

92Score (%)

51

Code Generation

LiveCodeBench

1,784Pass@1

51

LiveCodeBench Aug 24 – Jan 25

50.9Accuracy

45

Code Generation

LiveCodeBench v6

91.7Score

45

LiveCodeBench V3 V4 (test)

61.4Accuracy

42

Code Generation

LiveCodeBench (test)

53.6Pass@1 Overall

42

LiveCodeBench

70Accuracy

40

Code Generation

LiveCodeBench v5

61.5Pass@1

39

Code Generation

LiveCodeBench (LCB)

91.7Pass@1

37

Code Generation

LiveCodeBench

48Solve Rate

36

LiveCodeBench V5-6

50.8Accuracy

33

LiveCodeBench V1-4

47.1Accuracy

33