ALFRED
Benchmarks
TaskDatasetSOTA resultTrendResults
EB-ALFRED
82Average Score
87
ALFRED (test)
70Success Rate (SR)
74
ALFRED
19.84Accuracy
57
ALFRED EB
92Avg Score
36
ALFRED (test-unseen)
94.5GC
31
ALFRED
69.9Success Rate (SR)
28
ALFRED unseen (test)
4,572Success Rate
26
ALFRED seen (test)
53.23Success Rate (SR)
26
EB-ALFRED (Long)
74Success Rate (SR)
23
ALFRED seen 1.0 (test)
54.81GC
20
ALFRED (test unseen)
60.79Success Rate (SR)
18
ALFRED seen (test)
65.09Success Rate (SR)
18
Alfred
2.19pmae
15
ALFRED
78.01Object Perception (Grounding) Accuracy
12
ALFRED
45.81Success Rate (SR)
11
EB-ALFRED online unsupervised setting
61Success Rate (Avg)
10
ALFRED Open-ended
80.8Success Rate (SR)
8
ALFRED (Dynamic)
83.1Success Rate (SR)
8
ALFRED (Static)
88.4SR
8
ALFRED
62Accuracy
8
ALFRED unseen (val)
67.8Success Rate (SR)
8
ALFRED seen (test)
29.16Task Success Rate
7
ALFRED Object Shift [OS]
83.92F1 Score
7
ALFRED Template Shift [TS]
84.9F1 Score
7
ALFRED In-Distribution [ID]
84.28F1 Score
7