Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

APPS dataset and benchmarks · SOTA2 Research

Research Datasets

APPS

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

APPS

91.2Pass@1

111

Uncertainty Estimation

APPS Interview

81AUROC

79

Uncertainty Estimation

APPS Intro

0.813AUROC

79

Code Correctness Evaluation

APPS

80Accuracy

53

Leakage Detection

APPS

90.53Accuracy

42

Code Generation

APPS (test)

56.3Introductory Score

36

Code Generation

APPS Intermediate

81.95Pass Rate

32

Code Safety Evaluation

APPS 1.0 (test)

0.988Safety Score

30

Code Generation

APPS

45Accuracy

29

Code Generation

APPS Introductory

92.1pass@1

25

Code Generation

APPS Competition

38pass@1

20

Data leakage detection

APPS

81.91Accuracy

18

Code Generation

APPS Overall

21.38PR

18

Meta-reasoning quality assessment

APPS

85.6Thoroughness

12

Code Generation

APPS

83.2Pass@4

12

Program Synthesis

APPS 1.0 (test)

25.61Pass@5 (Introductory)

11

Code Generation

APPS

5.65Tau

10

Code Generation

APPS Interview

2.64Pass@1

9

Code Generation

APPS interview-level (test)

0.572Mean Score

8

Watermark message recovery

APPS-G

100Message Accuracy

8

Code Peak-Memory Prediction

APPS

0.96Correlation (rho)

7

Competitive Programming

APPS (val)

72.72Pass@1

6

APPS (test)

81.6pAUC

6

Code metric regression

APPS Leetcode (test)

0.474RMSE

6

Code Generation

APPS+

1.94Pass@1 (Introductory)

5

Page 1 of 2

PreviousNext

Popular tasks

Code Generation Uncertainty Estimation Program synthesis Dafny Code Synthesis Watermark Detection

Follow for updates

Join SOTA2 Contact