Multi-hop spatial reasoning on StepGame with distracting noise (test)
85.77k=1 AccuracyTP-MANN
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TP-MANNProtocol=Mean±Std over 5 runs2022.04 | 85.77 | 60.31 | 50.18 | 37.45 | 31.25 | 52.99 | |
| TPR-RNNProtocol=Mean±Std over 5 runs2022.04 | 70.29 | 46.03 | 36.14 | 26.82 | 24.77 | 40.81 | |
| STMProtocol=Mean±Std over 5 runs2022.04 | 53.42 | 35.96 | 23.03 | 18.45 | 15.14 | 29.2 | |
| UTProtocol=Mean±Std over 5 runs2022.04 | 45.11 | 28.36 | 17.41 | 14.07 | 13.45 | 23.68 | |
| RRNProtocol=Mean±Std over 5 runs2022.04 | 24.05 | 19.98 | 16.03 | 13.22 | 12.31 | 17.12 | |
| RNProtocol=Mean±Std over 5 runs2022.04 | 22.64 | 17.08 | 15.08 | 12.84 | 11.52 | 15.83 |