Vision-Language Navigation on TOUCHDOWN (dev)
1,948Task Completion Rate (TC)ARC
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ARCloss=L2STOP2020.09 | 1,948 | 17.05 | 19.02 | 55.68 | 18.97 | — | |
| ARC2020.09 | 1,533 | 18.61 | 14.62 | 48.56 | 14.48 | — | |
| RCONCAT2020.09 | 1,114 | 19.87 | 10.77 | 46.61 | 10.76 | — | |
| GA2020.09 | 985 | 21.43 | 9.5 | 46.86 | 9.44 | — | |
| FLAMEvisual context setting=restricted2024.08 | 41.28 | 9.14 | — | — | — | 55.96 | |
| VLN-Video (2024)visual context setting=broader2024.08 | 34.5 | 9.6 | — | — | — | — | |
| Loc4Plan (2024)visual context setting=broader2024.08 | 34.5 | 10.5 | — | — | — | — | |
| ORAR (2022)visual context setting=broader2024.08 | 30.05 | 11.12 | — | — | — | 45.5 | |
| VELMA (2024)visual context setting=broader2024.08 | 29.83 | 14.67 | — | — | — | 43.44 | |
| ORAR* (2022)visual context setting=restricted (consistent with FLAME)2024.08 | 20.99 | 10.97 | — | — | — | 30.78 | |
| ARC+L2S (2020)visual context setting=broader2024.08 | 19.48 | 17.05 | — | — | — | — | |
| Random2020.09 | 15 | 26.63 | 0.05 | 4.65 | 0.06 | — | |
| VLN-Trans (2021)visual context setting=broader2024.08 | 15 | 20.3 | — | — | — | 27 | |
| GA (2019)visual context setting=broader2024.08 | 12 | 18.7 | — | — | — | 25.2 | |
| RCONCAT (2019)visual context setting=broader2024.08 | 10.6 | 20.4 | — | — | — | 22.5 | |
| RCONCAT* (2019)visual context setting=restricted (consistent with FLAME)2024.08 | 9.27 | 19.14 | — | — | — | 16.4 | |
| GA* (2019)visual context setting=restricted (consistent with FLAME)2024.08 | 7.12 | 20.91 | — | — | — | 13.27 |