Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

APPS dataset and benchmarks – Page 2 · SOTA2 Research

Research Datasets

APPS

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

APPS+ Competition

2.67Pass@1

5

Coding Reasoning

Apps

68.3Pass Rate

5

Program Synthesis

APPS

25.61Pass@5 (Introductory)

5

Dafny Code Synthesis

APPS Vericoding-derived (test)

83Pass Rate

4

Code Generation

APPS (fixed 500-problem slice)

79.4Pass@4

4

Watermark Detection

APPS-G

99.64TPR

4

Code Generation

APPS

33.7Avg@8

4

Code Generation Oversight

APPS

63Safety Score

4

APPS (test)

21.7Strict Accuracy

4

Program Discrimination

APPS (test)

42.9Accuracy

4

Code-interpreter tasks

APPS verified

58.7Accuracy

3

Code Generation

APPS stdin-style Plus

83.4Syntax Validity

3

Code Generation

APPS-G

68.01Pass Rate Before

2

Dafny Code Synthesis

APPS Dafny-derived (val)

53.9Pass Rate

1

Page 2 of 2

Popular tasks

Code Generation Uncertainty Estimation Program synthesis Dafny Code Synthesis Watermark Detection

Follow for updates

Join SOTA2 Contact