# Navigation Instruction Generation on Unseen Routes 200 instances (test)

Source: [Navigation Instruction Generation on Unseen Routes 200 instances (test)](<https://www.sota2.com/research/sota/navigation-instruction-generation-on-unseen-routes-200-instances-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:48:35.207Z

Evaluation of Navigation Instruction Generation performance on the test split of the Unseen Routes 200 instances dataset, using metrics including BLEU, Length (Len.), Landmark (Landm.), SDTW, Success Rate (SR), and SNT.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Navigation Instruction Generation |
| Dataset (including any stated split) | Unseen Routes 200 instances (test) |
| Metric | Length (Len.) |
| Unit | SCALAR |
| Direction | Lower is better |
| Results loaded | 5 of 5 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Length (Len.)](<https://www.sota2.com/research/sota/navigation-instruction-generation-on-unseen-routes-200-instances-test.md?metric=Length%20(Len.)>): Lower is better; unit: SCALAR.

- [BLEU](<https://www.sota2.com/research/sota/navigation-instruction-generation-on-unseen-routes-200-instances-test.md?metric=BLEU>): Higher is better; unit: PERCENT.

- [Landmark Score (Landm.)](<https://www.sota2.com/research/sota/navigation-instruction-generation-on-unseen-routes-200-instances-test.md?metric=Landmark%20Score%20(Landm.)>): Higher is better; unit: SCALAR.

- [SDTW](<https://www.sota2.com/research/sota/navigation-instruction-generation-on-unseen-routes-200-instances-test.md?metric=SDTW>): Higher is better; unit: SCALAR.

- [Success Rate (SR)](<https://www.sota2.com/research/sota/navigation-instruction-generation-on-unseen-routes-200-instances-test.md?metric=Success%20Rate%20(SR)>): Higher is better; unit: PERCENT.

- [SNT Score](<https://www.sota2.com/research/sota/navigation-instruction-generation-on-unseen-routes-200-instances-test.md?metric=SNT%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Length (Len.) | Result date | Report paper |
| --- | --- | --- | --- |
| g2t+pretrain | 50.3 | Not recorded | 68fa6a011e209131cea2552c |
| graph2text | 50.5 | Not recorded | 68fa6a011e209131cea2552c |
| seq2seq | 51.8 | Not recorded | 68fa6a011e209131cea2552c |
| rule based | 52.3 | Not recorded | 68fa6a011e209131cea2552c |
| reference | 57.5 | Not recorded | 68fa6a011e209131cea2552c |

### g2t+pretrain

Length (Len.): 50.3. Result ID: 698dfcddffc3901de099f620.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a011e209131cea2552c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a011e209131cea2552c.md>)

[Implementation](<https://www.sota2.com/www.cl.uni-heidelberg.de/statnlpgroup/map2seq/>)

Recorded evaluation configuration: {"pretraining":true}. This record does not assert independent reproduction.

### graph2text

Length (Len.): 50.5. Result ID: 698dfcddffc3901de099f61f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a011e209131cea2552c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a011e209131cea2552c.md>)

[Implementation](<https://www.sota2.com/www.cl.uni-heidelberg.de/statnlpgroup/map2seq/>)

Recorded evaluation configuration: {"pretraining":false}. This record does not assert independent reproduction.

### seq2seq

Length (Len.): 51.8. Result ID: 698dfcddffc3901de099f61e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a011e209131cea2552c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa703d1e209131cea49db1.md>)

[Implementation](<https://github.com/tensorflow/tensor2tensor>)

Recorded evaluation configuration: {"architecture":"Transformer"}. This record does not assert independent reproduction.

### rule based

Length (Len.): 52.3. Result ID: 698dfcddffc3901de099f61d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a011e209131cea2552c.md>)

Recorded evaluation configuration: {"type":"heuristic system"}. This record does not assert independent reproduction.

### reference

Length (Len.): 57.5. Result ID: 698dfcddffc3901de099f61c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a011e209131cea2552c.md>)

Recorded evaluation configuration: {"type":"human reference"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.