Vision-and-Language Navigation on Room-to-Room (R2R) Unseen (val)
74.2SRPANOGEN
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PANOGEN2023.05 | 74.2 | 13.4 | 3.03 | 64.3 | |
| DUET-CLIPbackbone=CLIP-ViT-B/162023.05 | 72.8 | 12.92 | 3.19 | 63.4 | |
| DUETGraph-based navigation=true2023.04 | 72 | — | — | 60 | |
| VLN-SIGGraph-based navigation=true2023.04 | 72 | — | — | 62 | |
| DUETbackbone=ViT-B/162023.05 | 72 | 13.94 | 3.31 | 60 | |
| SE3DSenvironment augmentation=true2023.05 | 69 | — | 3.29 | 62 | |
| VLN-SIGGraph-based navigation=false2023.04 | 68 | — | — | 62 | |
| EnvEditenvironment augmentation=true, ensemble=false2023.05 | 67.9 | 12.13 | 3.22 | 62.9 | |
| HAMTGraph-based navigation=false2023.04 | 66 | — | — | 61 | |
| HAMT2023.05 | 66 | 11.46 | 2.29 | 61 | |
| HOPGraph-based navigation=false2023.04 | 64 | — | — | 57 | |
| REMenvironment augmentation=true2023.05 | 63.6 | 12.44 | 3.89 | 57.9 | |
| + FASTSearch Strategy=Beam, Trajectory length version=long2019.03 | 63 | 224.42 | 4.03 | 0.02 | |
| VLNBert (init. PREVALENT)protocol=single-run, pre-trained_encoders=true2021.07 | 63 | 12.01 | 3.93 | 0.57 | |
| OBERTGraph-based navigation=false2023.04 | 63 | — | — | 57 | |
| RecBERT2023.05 | 63 | 12.01 | 3.93 | 57 | |
| SSMGraph-based navigation=true2023.04 | 62 | — | — | 45 | |
| PREVALENTInstruction Setting=Multiple (M)2020.02 | 60 | 9.98 | 4.12 | 57 | |
| NvEMprotocol=single-run, pre-trained_encoders=false2021.07 | 60 | 11.83 | 4.27 | 0.55 | |
| PRESSInstruction Setting=Multiple (M)2020.02 | 59 | 10.06 | 4.31 | 55 | |
| VLNBert (init. OSCAR)protocol=single-run, pre-trained_encoders=true2021.07 | 59 | 11.86 | 4.29 | 0.53 | |
| PREVALENTInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 58 | 10.19 | 4.71 | 53 | |
| PREVALENTprotocol=single-run, pre-trained_encoders=true2021.07 | 58 | 10.19 | 4.71 | 0.53 | |
| PREVALENTGraph-based navigation=false2023.04 | 58 | — | — | 53 | |
| PREVALENT2023.05 | 58 | 10.19 | 4.71 | 53 | |
| SMNASearch Strategy=Beam2019.03 | 57 | — | 5.04 | — | |
| RelGraphprotocol=single-run, pre-trained_encoders=false2021.07 | 57 | 9.99 | 4.73 | 0.53 | |
| + FASTSearch Strategy=Greedy, Trajectory length version=short2019.03 | 56 | 21.17 | 4.97 | 0.43 | |
| FASTInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 56 | 21.17 | 4.97 | 43 | |
| AuxRNprotocol=single-run, pre-trained_encoders=false2021.07 | 55 | — | 5.28 | 0.5 | |
| ENVDROPInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 52 | 10.7 | 5.22 | 48 | |
| EnvDropprotocol=single-run, pre-trained_encoders=false2021.07 | 52 | 10.7 | 5.22 | 0.48 | |
| EnvDropenvironment augmentation=true2023.05 | 52 | 10.7 | 5.22 | 48 | |
| SPEAKER-FOLLOWERSearch Strategy=Beam2019.03 | 50 | — | 5.24 | — | |
| REGRETFULInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 50 | — | 5.32 | 41 | |
| Regretfulprotocol=single-run, pre-trained_encoders=false2021.07 | 50 | — | 5.32 | 0.41 | |
| VLNBert (no init.)protocol=single-run, pre-trained_encoders=false2021.07 | 50 | 10.31 | 5.1 | 0.46 | |
| PRESSInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 49 | 10.36 | 5.28 | 45 | |
| PRESSprotocol=single-run, pre-trained_encoders=true2021.07 | 49 | 10.36 | 5.28 | 0.45 | |
| Our baseline SMNA2019.03 | 47 | 12.61 | 5.48 | 0.41 | |
| SMprotocol=single-run, pre-trained_encoders=false2021.07 | 45 | — | 5.52 | 0.32 | |
| RCM+SIL(TRAIN)Instruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 43 | 11.46 | 6.09 | — | |
| RCM+SILprotocol=single-run, pre-trained_encoders=false2021.07 | 43 | 11.46 | 6.09 | — | |
| SPEAKER-FOLLOWERInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 35 | — | 6.62 | — | |
| Speaker-Followerprotocol=single-run, pre-trained_encoders=false2021.07 | 35 | — | 6.62 | — | |
| RPAInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 25 | — | 7.65 | — | |
| Seq2seq2019.03 | 22 | 8.39 | 7.81 | — | |
| SEQ2SEQInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 22 | 8.39 | 7.81 | — | |
| Seq2Seqprotocol=single-run, pre-trained_encoders=false2021.07 | 22 | 8.39 | 7.81 | — | |
| RANDOM2019.03 | 16 | 9.77 | 9.23 | — | |
| RANDOMInstruction Setting=Single (S), Decoding Strategy=Greedy2020.02 | 16 | 9.77 | 9.23 | — | |
| Randomprotocol=single-run, pre-trained_encoders=false2021.07 | 16 | 9.77 | 9.23 | — |