Instruction Generation on R2R (test)
76SRHumans (R2R dataset)
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Humans (R2R dataset)2022.12 | 76 | 67.6 | 71 | 64.8 | 14.2 | 3.6 | |
| EncDec-TransformerToM listener=None2022.12 | 54.7 | 43.8 | 49.4 | 40.4 | 15.8 | 3.4 | |
| EncDec-LSTMToM listener=10 VLNBERT2022.12 | 54.7 | 46 | 56.4 | 41.9 | 14 | 3.1 | |
| EncDec-TransformerToM listener=10 VLNBERT2022.12 | 52 | 44 | 54.2 | 41.6 | 17.7 | 3.2 | |
| EncDec-LSTMToM listener=None2022.12 | 49.3 | 37.6 | 45.3 | 33.8 | 17.4 | 3.3 | |
| Finetuned GPT-2ToM listener=10 VLNBERT2022.12 | 46.7 | 30.9 | 43.4 | 28.1 | 21.2 | 3 | |
| Finetuned GPT-2ToM listener=None2022.12 | 36 | 27.8 | 37.7 | 24.5 | 20.9 | 2.9 |