Navigation from Dialog History on NDH Unseen (val)
4.07Goal Progress (Oracle)HOP (2)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| HOP (2)pre-training=PREVALENT + BnB data, description=finetuned model pre-trained on data of both PREVALENT and processed data from BnB2022.03 | 4.07 | 4.05 | 4.41 | — | |
| HOP (1)pre-training=PREVALENT data, description=finetuned model pre-trained on the same data as PREVALENT2022.03 | 3.96 | 3.99 | 4.37 | — | |
| ORIST2022.03 | 3.3 | 3.29 | 3.55 | — | |
| HOP (0)pre-training=none, description=baseline model without pre-training2022.03 | 3.08 | 3.1 | 3.38 | — | |
| CMN2022.03 | 2.68 | 2.28 | 2.97 | — | |
| PREVALENT2022.03 | 2.58 | 2.99 | 3.15 | — | |
| Seq2Seq2022.03 | 1.23 | 1.98 | 2.1 | — | |
| AdapterUpdated Params(%)=3.082023.08 | — | — | — | 5.3 | |
| BitFitUpdated Params(%)=0.462023.08 | — | — | — | 3.77 | |
| Fine-TuningUpdated Params(%)=1002023.08 | — | — | — | 5.16 | |
| LoRAUpdated Params(%)=3.022023.08 | — | — | — | 5.16 | |
| Prompt TuningUpdated Params(%)=0.372023.08 | — | — | — | 3.26 | |
| VLN-PETLUpdated Params(%)=2.822023.08 | — | — | — | 5.69 |