Multi-hop spatial reasoning on StepGame larger k generalization (test)
28.53Accuracy (k=6)TP-MANN
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TP-MANNzero-shot generalization=true, trained_on=k=1 to 5, noise=distracting2022.04 | 28.53 | 26.45 | 23.67 | 22.52 | 21.46 | 24.53 | |
| TPR-RNNzero-shot generalization=true, trained_on=k=1 to 5, noise=distracting2022.04 | 22.25 | 19.88 | 15.45 | 13.01 | 12.65 | 16.65 | |
| STMzero-shot generalization=true, trained_on=k=1 to 5, noise=distracting2022.04 | 13.8 | 12.63 | 11.54 | 11.3 | 11.77 | 12.21 | |
| UTzero-shot generalization=true, trained_on=k=1 to 5, noise=distracting2022.04 | 12.73 | 12.11 | 11.4 | 11.41 | 11.74 | 11.88 | |
| RRNzero-shot generalization=true, trained_on=k=1 to 5, noise=distracting2022.04 | 11.62 | 11.4 | 11.83 | 11.22 | 11.69 | 11.56 | |
| RNzero-shot generalization=true, trained_on=k=1 to 5, noise=distracting2022.04 | 11.12 | 11.53 | 11.21 | 11.13 | 11.34 | 11.27 |