Navigation from Dialog History on NDH Unseen (test)
2.99Goal Progress (Oracle)HOP (2)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| HOP (2)pre-training=PREVALENT + BnB data, description=finetuned model pre-trained on data of both PREVALENT and processed data from BnB2022.03 | 2.99 | 3.18 | 3.24 | — | |
| HOP (1)pre-training=PREVALENT data, description=finetuned model pre-trained on the same data as PREVALENT2022.03 | 2.92 | 3.2 | 3.31 | — | |
| ORIST2022.03 | 2.78 | 3.17 | 3.15 | — | |
| CMN2022.03 | 2.69 | 2.26 | 2.95 | — | |
| HOP (0)pre-training=none, description=baseline model without pre-training2022.03 | 2.05 | 2.12 | 2.26 | — | |
| PREVALENT2022.03 | 1.67 | 2.39 | 2.44 | — | |
| Seq2Seq2022.03 | 1.25 | 2.11 | 2.35 | — | |
| AdapterUpdated Params(%)=3.082023.08 | — | — | — | 4.72 | |
| BitFitUpdated Params(%)=0.462023.08 | — | — | — | 4.03 | |
| Fine-TuningUpdated Params(%)=1002023.08 | — | — | — | 5.05 | |
| LoRAUpdated Params(%)=3.022023.08 | — | — | — | 5.91 | |
| Prompt TuningUpdated Params(%)=0.372023.08 | — | — | — | 2.86 | |
| VLN-PETLUpdated Params(%)=2.822023.08 | — | — | — | 6.13 |