VirtualHome
Benchmarks
TaskDatasetSOTA resultTrendResults
Virtualhome
63.3Success Rate
31
VirtualHome
73.5Overall Performance
24
VirtualHome 2.0 (test)
80Success Rate
19
VirtualHome v2.2.4 (Apartments #3 and #6)
38.41Speedup (%)
18
VirtualHome Unseen
36.9Success Rate
18
VirtualHome (Seen)
31.2Success Rate
18
VirtualHome
92Success Rate
16
VirtualHome Unseen domains
83.61Success Rate
15
VirtualHome
61.12SR
15
VirtualHome v2.3.0
41.9GCR
8
VirtualHome Open-ended
87.6Success Rate (SR)
8
VirtualHome Dynamic
85.4Success Rate
8
VirtualHome Static
90.1Success Rate (SR)
8
VirtualHome-Social Addition 4→5
48.213Average Steps
6
VirtualHome (unseen domains)
80.16Success Rate
6
VirtualHome average performance (unseen domains)
82.59SR
5
VirtualHome unseen domains 5-shot
83.61Success Rate
5
VirtualHome unseen domains 1-shot
81.56SR
5
VirtualHome Novel Apartment (Unseen)
82.9Simple Success Rate
4
VirtualHome livingroom_and_bedroom
82.7Task Success Rate (TSR)
3
VirtualHome kitchen_and_bedroom
81.9TSR
3
VirtualHome kitchen and bathroom
80.6TSR
3
VirtualHome bathroom_and_livingroom
81.9TSR
3
VirtualHome bedroom_and_bathroom
80.8TSR
3
VirtualHome kitchen and livingroom
80.8TSR
3