Navigation on REVERIE Unseen (test)
81.51SRHuman
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Human2022.03 | 81.51 | 86.83 | 53.66 | 21.18 | |
| Human2023.03 | 81.51 | 86.83 | 53.66 | 21.18 | |
| TANDEM2026.06 | 63.1 | — | 44.2 | — | |
| GOAT2026.06 | 57.7 | — | 40.5 | — | |
| ScaleVLN2026.06 | 56.1 | — | 39.5 | — | |
| AutoVLN2026.06 | 55.2 | — | 38.9 | — | |
| DUETMemory=Top. Map, Exploit=X2023.03 | 52.51 | 56.91 | 36.06 | 21.3 | |
| DUET2026.06 | 52.5 | — | 36.1 | — | |
| Meta-Explore*Memory=Top. Map, Exploit=local goal, dataset=modified (partially fixed object categories)2023.03 | 51.18 | 53.8 | 44.04 | 10.23 | |
| SAME2026.06 | 48.6 | — | 37.1 | — | |
| NaviLLM2026.06 | 39.8 | — | 32.3 | — | |
| Fine-TuningUpdated Params(%)=1002023.08 | 33.09 | 37.82 | 27.02 | 12.83 | |
| LORAUpdated Params(%)=3.332023.08 | 32.12 | 37 | 26.86 | 14.93 | |
| VLN-PETLUpdated Params(%)=2.812023.08 | 30.83 | 36.06 | 26.73 | 14 | |
| SIAMemory=Seq, Exploit=X2023.03 | 30.8 | 44.56 | 14.85 | 48.61 | |
| SIA2021.10 | 30.8 | 44.56 | 14.85 | 48.61 | |
| HAMTMemory=Seq, Exploit=X2023.03 | 30.4 | 33.41 | 26.67 | 13.62 | |
| HAMT2021.10 | 30.4 | 33.41 | 26.67 | 13.62 | |
| HAMT2026.06 | 30.4 | — | 26.7 | — | |
| AirBERT2022.03 | 30.28 | 34.2 | 23.61 | 17.91 | |
| AirbertMemory=Seq, Exploit=X2023.03 | 30.28 | 34.2 | 23.61 | 17.91 | |
| Rec (Airbert)mode=discriminative2021.08 | 30.28 | 34.2 | 23.61 | 17.91 | |
| HOPversion=22022.03 | 30.17 | 33.06 | 24.34 | 16.38 | |
| RecBERT2022.03 | 29.61 | 32.91 | 23.99 | 15.86 | |
| VLN BERTMemory=Rec, Exploit=X2023.03 | 29.61 | 32.91 | 23.99 | 15.86 | |
| RecBERT2021.10 | 29.61 | 32.91 | 23.99 | 15.86 | |
| AdapterUpdated Params(%)=3.392023.08 | 29.2 | 32.31 | 24.78 | 14.96 | |
| HOPversion=12022.03 | 29.12 | 32.26 | 23.37 | 17.05 | |
| Rec (OSCAR)mode=discriminative2021.08 | 24.62 | 26.67 | 19.48 | 14.88 | |
| Rec (VLN-BERT)mode=discriminative2021.08 | 23.57 | 26.83 | 18.73 | 17.63 | |
| HOPversion=02022.03 | 23.12 | 26.27 | 18.5 | 16.15 | |
| ORIST2022.03 | 22.19 | 29.2 | 18.97 | 11.38 | |
| Rec (ViLBERT)mode=discriminative2021.08 | 22.17 | 25.51 | 17.28 | 18.22 | |
| BitFitUpdated Params(%)=0.802023.08 | 21.5 | 24.95 | 18.85 | 12.53 | |
| Prompt TuningUpdated Params(%)=0.712023.08 | 19.95 | 24.24 | 17.94 | 11.61 | |
| FAST-MATTN2022.03 | 19.88 | 30.63 | 11.61 | 39.05 | |
| FAST-MATTNMemory=Rec, Exploit=X2023.03 | 19.88 | 30.63 | 11.61 | 39.05 | |
| FAST-MATTN2021.10 | 19.88 | 30.63 | 11.6 | 39.05 | |
| FAST-MATTN2021.08 | 19.88 | 30.63 | 11.61 | 39.05 | |
| FAST-Short2022.03 | 14.18 | 23.36 | 8.74 | 30.69 | |
| RCM2022.03 | 7.84 | 11.68 | 6.67 | 10.6 | |
| RCMMemory=Rec, Exploit=X2023.03 | 7.84 | 11.68 | 6.67 | 10.6 | |
| RCM2021.10 | 7.84 | 11.68 | 6.67 | 10.6 | |
| RCM2021.08 | 7.84 | 11.68 | 6.67 | 10.6 | |
| Seq2SeqMemory=Rec, Exploit=X2023.03 | 6.88 | 10.89 | 3.99 | 3.09 | |
| SMNA2022.03 | 5.8 | 8.39 | 4.53 | 9.23 | |
| SMNAMemory=Rec, Exploit=homing2023.03 | 5.8 | 8.39 | 4.53 | 9.23 | |
| SMNA2021.10 | 5.8 | 8.39 | 4.53 | 9.23 | |
| SMNA2021.08 | 5.8 | 8.39 | 4.53 | 9.23 | |
| Seq2Seq2021.10 | 3.99 | 6.88 | 3.09 | 10.89 | |
| Seq2Seq-SF2021.08 | 3.99 | 6.88 | 3.09 | 10.89 |