Continual Instruction Following on ALFRED
69.9Success Rate (SR)ExRAP
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ExRAPNon-stationarity=Low2025.09 | 69.9 | 5.94 | |
| ExRAPNon-stationarity=Medium2025.09 | 64 | 4.82 | |
| ExRAPNon-stationarity=High2025.09 | 59.11 | 4.42 | |
| LLM-PlannerNon-stationarity=Low2025.09 | 58.44 | 7.28 | |
| LLM-PlannerNon-stationarity=Medium2025.09 | 51.8 | 7.28 | |
| ProgPromptNon-stationarity=Low2025.09 | 47.15 | 9.81 | |
| SayCanNon-stationarity=Low2025.09 | 45.67 | 8.25 | |
| SayCanNon-stationarity=Medium2025.09 | 41.81 | 8.39 | |
| (SL)^3 + plannerData percentage=10%, Supervision level=Extra information, Modular components=Planner, Extra info details=Includes ground-truth training segmentations/alignments and ground-truth test-time plans2021.10 | 40.4 | — | |
| SayCanNon-stationarity=High2025.09 | 35.79 | 7.42 | |
| LLM-PlannerNon-stationarity=High2025.09 | 35.76 | 6.65 | |
| ProgPromptNon-stationarity=Medium2025.09 | 35.62 | 7.22 | |
| FILMSupervision level=Extra information, Extra info details=Includes ground-truth training segmentations/alignments and ground-truth test-time plans2021.10 | 20.1 | — | |
| ProgPromptNon-stationarity=High2025.09 | 19.97 | 7.52 | |
| ZSPNon-stationarity=Low2025.09 | 18.22 | 17.24 | |
| HLSMSupervision level=Goal + partial plan sup., Reproduction=Author's reproduction, Test-time info=Goals only2021.10 | 17.2 | — | |
| (SL)^3 + HLSMData percentage=10%, Supervision level=Goal + partial plan sup., Modular components=HLSM, Test-time info=Goals only2021.10 | 16.1 | — | |
| ZSPNon-stationarity=Medium2025.09 | 14.67 | 20.83 | |
| ABPSupervision level=Extra information, Extra info details=Includes ground-truth training segmentations/alignments and ground-truth test-time plans2021.10 | 12.6 | — | |
| HiTUTSupervision level=Extra information, Extra info details=Includes ground-truth training segmentations/alignments and ground-truth test-time plans2021.10 | 11.1 | — | |
| ZSPNon-stationarity=High2025.09 | 9.56 | 22.53 | |
| ETSupervision level=Extra information, Extra info details=Includes ground-truth training segmentations/alignments and ground-truth test-time plans2021.10 | 7.3 | — | |
| EmBERTSupervision level=Extra information, Extra info details=Includes ground-truth training segmentations/alignments and ground-truth test-time plans2021.10 | 5.7 | — | |
| MOCASupervision level=Extra information, Extra info details=Includes ground-truth training segmentations/alignments and ground-truth test-time plans2021.10 | 5.4 | — | |
| S+Supervision level=Extra information, Extra info details=Includes ground-truth training segmentations/alignments and ground-truth test-time plans2021.10 | 0.1 | — | |
| (SL)^3Data percentage=10%, Supervision level=Goal + partial plan sup., Test-time info=Goals only2021.10 | 0 | — | |
| seq2seqSupervision level=Goal + partial plan sup., Test-time info=Goals only2021.10 | 0 | — | |
| seq2seq2seqSupervision level=Goal + partial plan sup., Test-time info=Goals only2021.10 | 0 | — |