PHYRE
Benchmarks
TaskDatasetSOTA resultTrendResults
PHYRE DeepPHY (test)
14.92Att. 1 Score
36
PHYRE Set 3 (test)
52Pass@1
13
PHYRE Set 2 (test)
0.472Pass@1
13
PHYRE Set 1 (test)
52Pass@1
13
PHYRE-1B cross-template (test)
42.2AUCCESS
7
PHYRE-1B within-template (test)
86.2AUCCESS
7
PHYRE Cross-template 1.0
50.86Success Rate
6
PHYRE Within-template 1.0
85.49Success Rate (AUCCESS)
6
PHYRE-2B cross-template (test)
24.3AUCCESS
5
PHYRE cross-template B
56.31AUCCESS
5
PHYRE within-template B
85.49AUCCESS
5
PHYRE-C (test)
9.22Avg Prediction Error
4
PHYRE-W (t ∈ [T_train, 2 × T_train])
11.1Average Prediction Error
4
PHYRE-W (train)
1.31Avg Prediction Error
4
PHYRE cross-task generalization B-tier
42.2AUCCESS
3
PHYRE within-task generalization B-tier
85.2AUCCESS
3