AI2-THOR
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
AI2-THOR in-domain
67.3Accuracy
14
AI2-THOR in-domain
66.7Accuracy
14
AI2-THOR in-domain
97.8Accuracy
14
AI2-THOR (Evaluation Environments)
59.3SR
13
AI2-THOR (Simulations)
59.17Success Rate (SR)
12
AI2-THOR (test)
75SR
11
AI2-THOR Unseen Scenes (L >= 5) (test)
13.91SPL
11
AI2-THOR Offline Data 1 (test)
74.4CF-SR
8
AI2-THOR
78.35SR
8
AI2-THOR L ≥ 5 (val)
14.86SPL
8
AI2-THOR All (val)
0.173SPL
8
AI2-THOR
70.2Success Rate
7
AI2-THOR Unseen Scenes (All) (test)
16.15SPL
7
AI2-THOR Offline Data 2 (test)
60.5CF-SR
6
AI2-THOR Unseen Target Domains v1 (test)
86.4Success Rate (SR)
6
AI2-THOR Seen Target Domains v1 (test)
90.9Success Rate (SR)
6
AI2-THOR Source Domains v1 (train test)
97.9Success Rate (SR)
6
AI2-THOR Unseen Env., Unseen Goals 1.0
75Success Rate (SR)
5
AI2-THOR Unseen Env., Seen Goals 1.0
71Success Rate (SR)
5
AI2-THOR 1.0 (Seen Env., Unseen Goals)
73Success Rate (SR)
5
AI2-THOR Seen Env., Seen Goals 1.0
74Success Rate
5
AI2-THOR (test)
17.88SPL
4
AI2-THOR (test)
0.05LPIPS
3
AI2-THOR Imagined Scene (test)
271.8FVD
3
AI2-THOR ProcTHOR Habitat-compatible (unseen synthetic test environments)
58.1Success Rate (SR)
3