Question Answering on bAbI 1k (train test)
50Task: 1 Supporting Fact AccLSTM
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LSTM2015.03 | 50 | 80 | 80 | 39 | 30 | 52 | 51 | 55 | 36 | 56 | 38 | 26 | 6 | 73 | 79 | 77 | 49 | 48 | 92 | 9 | 51.3 | 20 | |
| MemN2NHops=2, Sentence Representation=PE, Linear Start=true, Joint Training=true2015.03 | 0.8 | 62 | 76.9 | 22.8 | 11 | 7.2 | 15.9 | 13.2 | 5.1 | 10.6 | 8.4 | 0.4 | 6.3 | 36.9 | 46.4 | 47.4 | 44.4 | 9.6 | 90.7 | 0 | 25.8 | 17 | |
| MemN2NHops=1, Sentence Representation=BoW, Joint Training=false2015.03 | 0.6 | 17.6 | 71 | 32 | 18.3 | 8.7 | 23.5 | 11.4 | 21.1 | 22.8 | 4.1 | 0.3 | 10.5 | 1.3 | 24.3 | 52 | 45.4 | 48.1 | 89.7 | 0.1 | 25.1 | 15 | |
| MemN2NHops=1, Sentence Representation=PE, Linear Start=true, Joint Training=false2015.03 | 0.2 | 12.8 | 58.8 | 11.6 | 15.7 | 8.7 | 20.3 | 12.7 | — | 18.6 | 0 | 0.1 | 0.3 | 2 | 0 | 1.6 | 49 | 10.1 | 85.6 | 0 | 16.3 | 12 | |
| MemNN WSH2015.03 | 0.1 | 42.8 | 76.4 | 40.3 | 16.3 | 51 | 36.1 | 37.8 | 35.9 | 68.7 | 30 | 10.1 | 19.7 | 18.3 | 64.8 | 50.5 | 50.9 | 51.3 | 100 | 3.6 | 40.2 | 18 | |
| MemN2NHops=1, Sentence Representation=PE, Joint Training=false2015.03 | 0.1 | 21.6 | 64.2 | 3.8 | 14.1 | 7.9 | 21.6 | 12.6 | 23.3 | 17.4 | 4.3 | 0.3 | 9.9 | 1.8 | 0 | 52.1 | 50.1 | 13.6 | 87.4 | 0 | 20.3 | 13 | |
| MemN2NHops=3, Sentence Representation=PE, Linear Start=true, Random Noise=true, Joint Training=true, Variation=Config A2015.03 | 0.1 | 14 | 33.1 | 5.7 | 14.8 | 3.3 | 17.9 | 10.1 | 3.1 | 6.6 | 0.9 | 0.3 | 1.4 | 8.2 | 0 | 3.5 | 44.5 | 9.2 | 90.2 | 0 | 13.3 | 11 | |
| MemN2NHops=3, Sentence Representation=PE, Linear Start=true, Layer-wise Weight Tying=true, Joint Training=true2015.03 | 0.1 | 18.8 | 31.7 | 17.5 | 12.9 | 2 | 10.1 | 6.1 | 1.5 | 2.6 | 3.3 | 0 | 0.5 | 2 | 1.8 | 51 | 42.6 | 9.2 | 90.6 | 0.2 | 15.2 | 10 | |
| Strongly Supervised MemNNSupervision=Strongly Supervised2015.03 | 0 | 0 | 0 | 0 | 2 | 0 | 15 | 9 | 0 | 2 | 0 | 0 | 0 | 1 | 0 | 0 | 35 | 5 | 64 | 0 | 6.7 | 4 | |
| MemN2NHops=1, Sentence Representation=PE, Linear Start=true, Random Noise=true, Joint Training=false2015.03 | 0 | 8.3 | 40.3 | 2.8 | 13.1 | 7.6 | 17.3 | 10 | 13.2 | 15.1 | 0.9 | 0.2 | 0.4 | 1.7 | 0 | 1.3 | 51 | 11.1 | 82.8 | 0 | 13.9 | 11 | |
| MemN2NHops=3, Sentence Representation=PE, Linear Start=true, Joint Training=true2015.03 | 0 | 15.6 | 31.6 | 2.2 | 13.4 | 2.3 | 25.4 | 11.7 | 2 | 5 | 1.2 | 0 | 0.2 | 8.1 | 0.5 | 51.3 | 41.2 | 10.3 | 89.9 | 0.1 | 15.6 | 11 | |
| MemN2NHops=3, Sentence Representation=PE, Linear Start=true, Random Noise=true, Joint Training=true, Variation=Config B2015.03 | 0 | 11.4 | 21.9 | 13.4 | 14.4 | 2.8 | 18.3 | 9.3 | 1.9 | 6.5 | 0.3 | 0.1 | 0.2 | 6.9 | 0 | 2.7 | 40.4 | 9.4 | 88 | 0 | 12.4 | 11 |