APPS
Benchmarks
TaskDatasetSOTA resultTrendResults
APPS+ Competition
2.67Pass@1
5
Apps
68.3Pass Rate
5
APPS
25.61Pass@5 (Introductory)
5
APPS Vericoding-derived (test)
83Pass Rate
4
APPS (fixed 500-problem slice)
79.4Pass@4
4
APPS-G
99.64TPR
4
APPS
33.7Avg@8
4
APPS
63Safety Score
4
APPS (test)
21.7Strict Accuracy
4
APPS (test)
42.9Accuracy
4
APPS verified
58.7Accuracy
3
APPS stdin-style Plus
83.4Syntax Validity
3
APPS-G
68.01Pass Rate Before
2
APPS Dafny-derived (val)
53.9Pass Rate
1