Question Answering on bAbI 1.0 (test)
0.4Task 1 Accuracybi-GRU
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bi-GRUcoreference features=one-hot2017.03 | 0.4 | 65.7 | 51.8 | 0.3 | 1.3 | 33.9 | 22.4 | 24.3 | 23.7 | 17.8 | 0.5 | 0.9 | 0.7 | 21.6 | 46.4 | 51 | 42.7 | 9.9 | 80.5 | 2.3 | 24.9 | 13 | |
| N2N2017.03 | 0.1 | 18.8 | 31.7 | 17.5 | 12.9 | 2 | 10.1 | 6.1 | 1.5 | 2.6 | 3.3 | 0 | 0.5 | 2 | 1.8 | 51 | 42.6 | 9.2 | 90.6 | 0.2 | 15.2 | 10 | |
| QRN2017.03 | 0 | 0.7 | 5.7 | 0 | 1.1 | 0.9 | 9.6 | 5.6 | 0 | 0 | 0 | 0 | 0 | 0.8 | 0 | 53 | 34.4 | 7.9 | 78.7 | 0.2 | 9.9 | 7 | |
| bi-GRU2017.03 | 0 | 59.7 | 51.1 | 0.5 | 2.4 | 21.5 | 18.7 | 23.3 | 23.4 | 21.8 | 0.6 | 0.4 | 0.8 | 20.5 | 50.4 | 51.5 | 35.7 | 8.8 | 81.5 | 1.8 | 23.7 | 13 | |
| MAGE-GRUbackbone=bi-GRU, coreference state dimensions=162017.03 | 0 | 9.7 | 1.5 | 0 | 1.5 | 3 | 12 | 11.9 | 1.2 | 0.2 | 0 | 0.1 | 1.9 | 9.7 | 27.3 | 45 | 33.8 | 8.3 | 37.7 | 0.2 | 10.3 | 9 | |
| GA2017.03 | 0 | 62.4 | 47.3 | 0.8 | 1.4 | 18.3 | 16.9 | 23.7 | 20.5 | 20.2 | 0 | 0 | 0.6 | 23.9 | 51.3 | 50.8 | 42.1 | 7.1 | 91.3 | 3.6 | 24.1 | 13 | |
| GAcoreference features=one-hot2017.03 | 0 | 66.4 | 49.3 | 0.5 | 1.1 | 21 | 16 | 15.6 | 25.8 | 33.6 | 0.2 | 0 | 0.9 | 22.6 | 52 | 50.7 | 39.6 | 10 | 90.5 | 3.9 | 25 | 13 | |
| GA + DAG-RNNbackbone=GA2017.03 | 0 | 22.9 | 46.3 | 0.4 | 2.5 | 9.1 | 7.5 | 46.4 | 0.8 | 0.6 | 2 | 3.4 | 5.9 | 19.5 | 63.3 | 51.4 | 46 | 7.4 | 92.1 | 7.8 | 21.8 | 13 | |
| MAGE-RNNbackbone=GA, weight sharing strategy=shared2017.03 | 0 | 1.2 | 2.6 | 0 | 0.9 | 1.7 | 6.3 | 9.6 | 0.1 | 0.7 | 0 | 0.1 | 0 | 19.7 | 34.7 | 3 | 43.3 | 9.9 | 62.8 | 0.1 | 9.8 | 7 | |
| MAGE-RNNbackbone=GA, coreference state dimensions=162017.03 | 0 | 1 | 1.5 | 0 | 0.6 | 0.5 | 4.8 | 2.5 | 0.2 | 0.4 | 0.1 | 0 | 0.6 | 13.8 | 18.4 | 2.4 | 43.2 | 5.5 | 76 | 2.4 | 8.7 | 5 |