Vision-and-Language Navigation on Room-to-Room (R2R) Seen (val)
2.9NE (Navigation Error)VLNBert (init. PREVALENT)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| VLNBert (init. PREVALENT)protocol=single-run, pre-trained_encoders=true2021.07 | 2.9 | 11.13 | 72 | 0.68 | |
| PRESSInstruction Setting=Multiple (M)2020.02 | 3.09 | 10.35 | 71 | 67 | |
| VLNBert (init. OSCAR)protocol=single-run, pre-trained_encoders=true2021.07 | 3.11 | 10.79 | 71 | 0.67 | |
| + FASTSearch Strategy=Beam, Trajectory length version=long2019.03 | 3.13 | 188.06 | 70 | 0.04 | |
| SMprotocol=single-run, pre-trained_encoders=false2021.07 | 3.22 | — | 67 | 0.58 | |
| SMNASearch Strategy=Beam2019.03 | 3.23 | — | 70 | — | |
| REGRETFULInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 3.23 | — | 69 | 63 | |
| Regretfulprotocol=single-run, pre-trained_encoders=false2021.07 | 3.23 | — | 69 | 0.63 | |
| Our baseline SMNA2019.03 | 3.31 | 11.69 | 69 | 0.63 | |
| PREVALENTInstruction Setting=Multiple (M)2020.02 | 3.31 | 10.31 | 67 | 63 | |
| AuxRNprotocol=single-run, pre-trained_encoders=false2021.07 | 3.33 | — | 70 | 0.67 | |
| SPEAKER-FOLLOWERInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 3.36 | — | 66 | — | |
| Speaker-Followerprotocol=single-run, pre-trained_encoders=false2021.07 | 3.36 | — | 66 | — | |
| NvEMprotocol=single-run, pre-trained_encoders=false2021.07 | 3.44 | 11.09 | 69 | 0.65 | |
| RelGraphprotocol=single-run, pre-trained_encoders=false2021.07 | 3.47 | 10.13 | 67 | 0.65 | |
| RCM+SIL(TRAIN)Instruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 3.53 | 10.65 | 67 | — | |
| RCM+SILprotocol=single-run, pre-trained_encoders=false2021.07 | 3.53 | 10.65 | 67 | — | |
| PREVALENTInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 3.67 | 10.32 | 69 | 65 | |
| PREVALENTprotocol=single-run, pre-trained_encoders=true2021.07 | 3.67 | 10.32 | 69 | 0.65 | |
| SPEAKER-FOLLOWERSearch Strategy=Beam2019.03 | 3.88 | — | 63 | — | |
| VLNBert (no init.)protocol=single-run, pre-trained_encoders=false2021.07 | 3.92 | 9.78 | 62 | 0.59 | |
| ENVDROPInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 3.99 | 11 | 62 | 59 | |
| EnvDropprotocol=single-run, pre-trained_encoders=false2021.07 | 3.99 | 11 | 62 | 0.59 | |
| PRESSInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 4.39 | 10.57 | 58 | 55 | |
| PRESSprotocol=single-run, pre-trained_encoders=true2021.07 | 4.39 | 10.57 | 58 | 0.55 | |
| RPAInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 5.56 | — | 43 | — | |
| Seq2seq2019.03 | 6.01 | 11.33 | 39 | — | |
| SEQ2SEQInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 6.01 | 11.33 | 39 | — | |
| Seq2Seqprotocol=single-run, pre-trained_encoders=false2021.07 | 6.01 | 11.33 | 39 | — | |
| RANDOMInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 9.15 | 9.58 | 16 | — | |
| RANDOM2019.03 | 9.45 | 9.58 | 16 | — | |
| Randomprotocol=single-run, pre-trained_encoders=false2021.07 | 9.45 | 9.58 | 16 | — |