Instruction Following on ALFRED (test-unseen)
94.5GCHuman
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Human2026.05 | 94.5 | 87.6 | 85.8 | 91 | |
| HELPER + ODIN2024.01 | 61.6 | — | — | 47.7 | |
| HELPER2024.01 | 55 | — | — | 37.4 | |
| OPEXInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 54.18 | 15.27 | 1,348 | 41.85 | |
| OPEX-SInstruction Level=High-level goal instructions only2024.03 | 53.82 | 18.46 | 1,257 | 41.27 | |
| FILM2024.01 | 42.9 | — | — | 30.7 | |
| EPAInstruction Level=High-level goal instructions only2024.03 | 39.54 | 3.91 | 292 | 36.07 | |
| LGS-RPAInstruction Level=High-level goal instructions only2024.03 | 38.55 | 20.01 | 1,292 | 27.8 | |
| FILMStep-by-step instructions=true2024.03 | 38.52 | — | — | 27.8 | |
| FILMInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 36.37 | 14.3 | 1,055 | 26.49 | |
| FILMInstruction Level=High-level goal instructions only2024.03 | 34.75 | 13.13 | 967 | 24.46 | |
| Scene-LLMStep-by-step instructions=false2024.03 | 33.75 | — | — | 25.15 | |
| HLSMInstruction Level=High-level goal instructions only2024.03 | 27.24 | 8.45 | 434 | 16.29 | |
| HLSMStep-by-step instructions=false2024.03 | 27.24 | — | — | 20.27 | |
| LLM-PlannerStep-by-step instructions=true2024.03 | 23.37 | — | — | 16.42 | |
| LLM-PlannerInstruction Level=High-level goal instructions only2024.03 | 22.89 | — | — | 13.41 | |
| LLM-PlannerStep-by-step instructions=false2024.03 | 22.89 | — | — | 13.41 | |
| LWITInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 20.91 | 16.34 | 560 | 9.42 | |
| Ours (multiple views)View Configuration=multiple views2026.05 | 19.13 | 14.81 | 5.06 | 8.37 | |
| E.T.Instruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 18.56 | 11.46 | 410 | 8.57 | |
| E.T.Step-by-step instructions=true2024.03 | 18.56 | — | — | 8.57 | |
| HITUTStep-by-step instructions=false2024.03 | 17.89 | — | — | 11.12 | |
| LAVInstruction Level=High-level goal instructions only2024.03 | 17.27 | 10.47 | 312 | 6.38 | |
| Ours (single view)View Configuration=single view2026.05 | 14.71 | 10.88 | 2.37 | 4.45 | |
| MOCAInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 14.28 | 9.99 | 272 | 5.3 | |
| MOCA2026.05 | 14.28 | 9.99 | 2.72 | 5.3 | |
| Ours (winning entry)View Configuration=winning entry2026.05 | 12.34 | 9.44 | 2.24 | 4.45 | |
| Singh et al.2026.05 | 8.08 | 5.2 | 0.7 | 1.5 | |
| Legg et al.2026.05 | 7.68 | 4.31 | 0.36 | 0.85 | |
| Seq2SeqInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 7.03 | 4.26 | 8 | 3.9 | |
| Shridhar et al.2026.05 | 7.03 | 4.26 | 0.8 | 0.39 |