List your company

ResearchFormerly WizWand

APPS

Benchmarks

TaskDatasetSOTA resultTrendResults

Code Generation

APPS

91.2Pass@1

111

Uncertainty Estimation

APPS Interview

81AUROC

79

Uncertainty Estimation

APPS Intro

0.813AUROC

79

Code Correctness Evaluation

APPS

80Accuracy

53

Leakage Detection

APPS

90.53Accuracy

42

Code Generation

APPS (test)

56.3Introductory Score

36

Code Generation

APPS Intermediate

81.95Pass Rate

32

Code Safety Evaluation

APPS 1.0 (test)

0.988Safety Score

30

Code Generation

APPS

45Accuracy

29

Code Generation

APPS Introductory

92.1pass@1

25

Code Generation

APPS Competition

38pass@1

20

Data leakage detection

APPS

81.91Accuracy

18

Code Generation

APPS Overall

21.38PR

18

Meta-reasoning quality assessment

APPS

85.6Thoroughness

12

Code Generation

APPS

83.2Pass@4

12

Program Synthesis

APPS 1.0 (test)

25.61Pass@5 (Introductory)

11

Code Generation

APPS

5.65Tau

10

Code Generation

APPS Interview

2.64Pass@1

9

Code Generation

APPS interview-level (test)

0.572Mean Score

8

Watermark message recovery

APPS-G

100Message Accuracy

8

Code Peak-Memory Prediction

APPS

0.96Correlation (rho)

7

Competitive Programming

APPS (val)

72.72Pass@1

6

APPS (test)

81.6pAUC

6

Code metric regression

APPS Leetcode (test)

0.474RMSE

6

Code Generation

APPS+

1.94Pass@1 (Introductory)

5