Vision-Language Navigation on R4R (val seen)
66.8SRVLN↻Bert+REM
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| VLN↻Bert+REMsetting=single-run2021.06 | 66.8 | 3.77 | — | 56.8 | 61.5 | 41.5 | 57.4 | |
| SSM2021.03 | 63 | 4.6 | 19.4 | 65 | 0.56 | 0.44 | — | |
| PTAlevel=high-level2021.03 | 58 | 4.54 | 16.5 | 60 | 0.58 | 0.41 | — | |
| PTAsetting=single-run2021.06 | 58 | 4.53 | — | 60 | 58 | 41 | 39 | |
| PTAlevel=low-level2021.03 | 57 | 5.11 | 11.9 | 52 | 0.42 | 0.29 | — | |
| RCMoptimization=goal oriented2021.03 | 56 | 5.11 | 24.5 | 40 | — | — | — | |
| OAAM2021.03 | 56 | — | 11.8 | 54 | — | 0.32 | — | |
| OAAM2022.04 | 56 | — | — | 54 | — | 0.32 | 49 | |
| VLN↻Bertsetting=single-run, reproduced=true2021.06 | 55.7 | 4.84 | — | 47.8 | 55.8 | 37.9 | 46 | |
| RelGraph2022.04 | 55 | 5.14 | — | 51 | 0.48 | 0.35 | 50 | |
| NvEM2022.04 | 54 | 5.38 | — | 51 | 0.48 | 0.35 | 47 | |
| RCMoptimization=fidelity oriented2021.03 | 53 | 5.37 | 18.8 | 55 | — | — | — | |
| RCMsetting=single-run2021.06 | 52.6 | 5.37 | — | 55.3 | — | — | 30.6 | |
| Speaker-Follower2021.03 | 52 | 5.35 | 15.4 | 46 | — | — | — | |
| E-Drop2021.03 | 52 | — | 19.9 | 53 | — | 0.27 | — | |
| EnvDrop2022.04 | 52 | — | — | 53 | — | 0.27 | 41 | |
| Speaker-Followersetting=single-run2021.06 | 51.9 | 5.35 | — | 46.4 | — | — | 37.3 | |
| NvEM+SEvol2022.04 | 50 | 5.77 | — | 48 | 0.45 | 0.3 | 40 | |
| EnvDrop+REMsetting=single-run2021.06 | 46.3 | 5.83 | — | 45.1 | 49.7 | 33.4 | 43.5 | |
| EnvDropsetting=single-run, reproduced=true2021.06 | 42.7 | 5.94 | — | 40.2 | 41.8 | 29.6 | 39.5 | |
| IL+RL+REMsetting=single-run2021.06 | 39.9 | 6.72 | — | 42.4 | 47.3 | 31.2 | 36.5 | |
| IL+RLsetting=single-run, reproduced=true2021.06 | 35.3 | 5.94 | — | 37.1 | 38.7 | 26.5 | 32.5 |