Embodied Task Completion on ALFRED seen (test)
53.23Success Rate (SR)Prompter
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Prompter2023.06 | 53.23 | 64.43 | 25.81 | 30.72 | |
| RePlan-BotInstruction Type=Task Description + Step-by-step Instructions2026.05 | 52.05 | 61.21 | 24.6 | 26.69 | |
| PrompterInstruction Type=Task Description + Step-by-step Instructions2026.05 | 51.17 | 60.22 | 25.12 | 30.21 | |
| RePlan-BotInstruction Type=Task Description Only2026.05 | 49.23 | 57.81 | 23.37 | 25.23 | |
| PrompterInstruction Type=Task Description Only2026.05 | 47.95 | 56.98 | 23.29 | 28.42 | |
| CAPEAMInstruction Type=Task Description + Step-by-step Instructions2026.05 | 47.61 | 57.32 | 22.82 | 26.31 | |
| ABPInstruction Type=Task Description + Step-by-step Instructions2026.05 | 44.55 | 51.13 | 3.88 | 4.92 | |
| CAPEAMInstruction Type=Task Description Only2026.05 | 43.64 | 53.14 | 19.76 | 24.32 | |
| LGS-RPA2023.06 | 40.05 | 48.66 | 21.28 | 28.97 | |
| LGS-RPAInstruction Type=Task Description + Step-by-step Instructions2026.05 | 40.05 | 48.66 | 21.28 | 28.97 | |
| EPA2023.06 | 39.96 | 44.14 | 2.56 | 3.47 | |
| ET2023.06 | 38.42 | 45.44 | 27.78 | 34.93 | |
| E.T.Step-by-step instructions=true2024.03 | 38.42 | 45.44 | — | — | |
| LGS-RPAInstruction Type=Task Description Only2026.05 | 33.01 | 41.71 | 16.65 | 24.49 | |
| FILM2023.06 | 28.83 | 39.55 | 11.27 | 15.59 | |
| FILMStep-by-step instructions=true2024.03 | 28.83 | 39.55 | — | — | |
| FILMInstruction Type=Task Description + Step-by-step Instructions2026.05 | 28.83 | 39.55 | 11.27 | 15.59 | |
| Scene-LLMStep-by-step instructions=false2024.03 | 26.52 | 37.09 | — | — | |
| FILMInstruction Type=Task Description Only2026.05 | 25.77 | 36.15 | 10.39 | 14.17 | |
| HLSM2023.06 | 25.11 | 35.15 | 10.39 | 14.17 | |
| HLSMStep-by-step instructions=false2024.03 | 25.11 | 35.79 | — | — | |
| HLSMInstruction Type=Task Description Only2026.05 | 25.11 | 35.79 | 6.69 | 11.53 | |
| LLM-PlannerStep-by-step instructions=true2024.03 | 18.2 | 26.77 | — | — | |
| LLM-PlannerStep-by-step instructions=false2024.03 | 15.33 | 24.57 | — | — | |
| HITUTStep-by-step instructions=false2024.03 | 13.63 | 21.11 | — | — | |
| Seq2Seq2023.06 | 3.98 | 9.42 | 2.02 | 6.27 |