Embodied Instruction Following on ALFRED seen 1.0 (test)
54.81GCOPEX
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OPEXInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 54.81 | 22.08 | 14.52 | 44.03 | |
| OPEx-SInstruction type=High-level goal instructions only, Perception Models=Fine-tuned ZoeDepth & SOLQ2024.03 | 54.27 | 20.13 | 13.64 | 43.51 | |
| OPEX-SInstruction Level=High-level goal instructions only2024.03 | 54.27 | 20.13 | 13.64 | 43.51 | |
| E.T.Instruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 45.44 | 34.93 | 27.78 | 38.42 | |
| EPAInstruction type=High-level goal instructions only2024.03 | 44.14 | 3.47 | 2.56 | 39.96 | |
| EPAInstruction Level=High-level goal instructions only2024.03 | 44.14 | 3.47 | 2.56 | 39.96 | |
| LGS-RPAInstruction type=High-level goal instructions only2024.03 | 41.71 | 24.49 | 16.65 | 33.01 | |
| LGS-RPAInstruction Level=High-level goal instructions only2024.03 | 41.71 | 24.49 | 16.65 | 33.01 | |
| LWITInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 40.53 | 23.1 | 43.1 | 30.92 | |
| FILMInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 38.51 | 15.06 | 11.23 | 27.67 | |
| FILMInstruction type=High-level goal instructions only2024.03 | 36.15 | 14.17 | 10.39 | 25.77 | |
| FILMInstruction Level=High-level goal instructions only2024.03 | 36.15 | 14.17 | 10.39 | 25.77 | |
| HLSMInstruction type=High-level goal instructions only2024.03 | 35.79 | 11.53 | 6.69 | 25.11 | |
| HLSMInstruction Level=High-level goal instructions only2024.03 | 35.79 | 11.53 | 6.69 | 25.11 | |
| MOCAInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 28.29 | 22.05 | 15.1 | 22.05 | |
| LLM-PlannerInstruction type=High-level goal instructions only2024.03 | 24.57 | — | — | 15.33 | |
| LLM-PlannerInstruction Level=High-level goal instructions only2024.03 | 24.57 | — | — | 15.33 | |
| LAVInstruction type=High-level goal instructions only2024.03 | 23.21 | 13.18 | 6.31 | 13.35 | |
| LAVInstruction Level=High-level goal instructions only2024.03 | 23.21 | 13.18 | 6.31 | 13.35 | |
| Seq2SeqInstruction Level=High-level Goal Instruction + Low-level step-by-step instructions2024.03 | 9.42 | 6.27 | 2.02 | 3.98 |