R2R
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
R2R-CE (val-unseen)
81.1Success Rate (SR)
779
R2R (val unseen)
84Success Rate (SR)
476
R2R (test unseen)
86SR
162
R2R (val seen)
7,540Success Rate (SR)
150
R2R Unseen (test)
86SR
144
R2R (val seen)
83.74Success Rate (SR)
68
R2R-CE (test-unseen)
66SR
63
R2R (test)
77SPL (Success weighted Path Length)
51
R2R unseen v1.0 (val)
3,130SR
48
R2R 1 (test unseen)
0.76Success Rate
29
R2R-Goal (val unseen)
36Success Rate (SR)
22
R2R-CE
4.73Navigation Error (NE)
19
R2R VLN-PE (val unseen)
4.33Navigation Error (NE)
18
R2R VLN-PE (val seen)
4.1Navigation Error (NE)
17
R2R Unseen (val)
4.42Navigation Error (NE)
16
R2R VLN Challenge Leaderboard (test)
1,257.38PL
16
R2R-Goal (test)
33BLEU-4
13
R2R-Goal (val (Seen))
36BLEU-4
13
R2R (Val-U)
66SPL
13
R2R
43.7Success Rate (SR)
12
R2R Discrete (val-unseen)
2.09Navigation Error (NE)
12
R2R unseen (test)
62.2Success Rate (SR)
11
R2R 1 (val seen)
1.67Navigation Error (NE)
10
R2R Unseen House (val)
4.83Navigation Error (NE)
9
R2R generalization (unseen)
35.2SR
8