Vision-and-Language Navigation on Room-across-Room (RxR) unseen (test)
93.9SRHuman
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Human2022.10 | 93.9 | 79.5 | 76.9 | 0.9 | |
| MARVALTraining protocol=Pre-Explore, Privileged access=Yes2022.10 | 61.8 | 68.6 | 54.8 | 5.2 | |
| MARVALTraining protocol=DAGGER2022.10 | 60.7 | 66.8 | 53.5 | 5.5 | |
| EnvEdit (on HAMT)Pre-trained=true, Single-run setting=true2022.03 | 60.4 | 64.6 | 51.8 | — | |
| EnvEditEnsemble=3 agents2022.10 | 60.4 | 64.6 | 51.8 | 5.1 | |
| HAMTPre-trained=true, Single-run setting=true2022.03 | 53.1 | 59.9 | 45.2 | — | |
| HAMT2022.10 | 53.1 | 59.9 | 45.2 | 6.2 | |
| EnvEdit (on EnvDrop-CLIP)Pre-trained=false, Single-run setting=true2022.03 | 46.2 | 56.4 | 40.4 | — | |
| CLEAR-C2022.10 | 40.3 | 53.7 | 34.9 | — | |
| EnvDrop + CLIP-VILSetup=mono-lingual2021.07 | 38.3 | 51.1 | — | — | |
| EnvDrop-CLIPPre-trained=false, Single-run setting=true2022.03 | 38.3 | 51.1 | 32.4 | — | |
| EnvDrop+2022.10 | 38.3 | 51.1 | 32.4 | — | |
| SAASetup=mono-lingual2021.07 | 35.4 | 46.8 | — | — | |
| Mono-BaselineSetup=mono-lingual2021.07 | 25.4 | 41.1 | — | — | |
| LSTM2022.10 | 21 | 36.8 | 16.9 | 12 | |
| Random-BaselineSetup=mono-lingual2021.07 | 7.5 | 15.4 | — | — |