Embodied Task Completion on ALFRED unseen (test)
4,572Success RatePrompter
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Prompter2023.06 | 4,572 | 58.76 | 2,076 | 26.22 | |
| EPA2023.06 | 3,607 | 39.54 | 292 | 3.91 | |
| LGS-RPA2023.06 | 3,541 | 45.24 | 1,568 | 22.76 | |
| FILM2023.06 | 2,780 | 38.52 | 1,132 | 15.13 | |
| HLSM2023.06 | 2,446 | 34.75 | 967 | 13.13 | |
| ET2023.06 | 857 | 18.56 | 410 | 11.46 | |
| RePlan-BotInstruction Type=Task Description + Step-by-step Instructions2026.05 | 47.61 | 60.29 | 21.89 | 26.31 | |
| PrompterInstruction Type=Task Description + Step-by-step Instructions2026.05 | 45.32 | 56.57 | 20.79 | 25.8 | |
| RePlan-BotInstruction Type=Task Description Only2026.05 | 44.79 | 57.02 | 21.89 | 26.31 | |
| CAPEAMInstruction Type=Task Description + Step-by-step Instructions2026.05 | 43.17 | 56.52 | 20.13 | 24.26 | |
| PrompterInstruction Type=Task Description Only2026.05 | 41.53 | 56.57 | 18.84 | 25.8 | |
| OPEx-SInstruction type=High-level goal instructions only, Perception Models=Fine-tuned ZoeDepth & SOLQ2024.03 | 41.27 | 53.82 | 12.57 | 18.46 | |
| CAPEAMInstruction Type=Task Description Only2026.05 | 40.42 | 54.17 | 19.34 | 23.68 | |
| Seq2Seq2023.06 | 39 | 7.03 | 8 | 4.26 | |
| EPAInstruction type=High-level goal instructions only2024.03 | 36.07 | 39.54 | 2.92 | 3.91 | |
| LGS-RPAInstruction Type=Task Description + Step-by-step Instructions2026.05 | 35.41 | 45.24 | 22.76 | 22.76 | |
| LGS-RPAInstruction type=High-level goal instructions only2024.03 | 27.8 | 38.55 | 12.92 | 20.01 | |
| LGS-RPAInstruction Type=Task Description Only2026.05 | 27.8 | 38.55 | 12.92 | 20.01 | |
| FILMInstruction Type=Task Description + Step-by-step Instructions2026.05 | 27.8 | 38.52 | 11.32 | 15.13 | |
| FILMInstruction type=High-level goal instructions only2024.03 | 24.46 | 34.75 | 9.67 | 13.13 | |
| FILMInstruction Type=Task Description Only2026.05 | 24.46 | 34.75 | 9.67 | 13.13 | |
| HLSMInstruction type=High-level goal instructions only2024.03 | 16.29 | 27.24 | 4.34 | 8.45 | |
| HLSMInstruction Type=Task Description Only2026.05 | 16.29 | 27.24 | 4.34 | 8.45 | |
| ABPInstruction Type=Task Description + Step-by-step Instructions2026.05 | 15.43 | 24.76 | 1.08 | 2.22 | |
| LLM-PlannerInstruction type=High-level goal instructions only2024.03 | 13.41 | 22.89 | — | — | |
| LAVInstruction type=High-level goal instructions only2024.03 | 6.38 | 17.27 | 3.12 | 10.47 |