Question Answering on bAbI 10k (test)
0Task 1: 1 Supporting Fact ErrorMemN2N
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MemN2NTraining samples=10k2016.12 | 0 | 0.3 | 2.1 | 0 | 0.8 | 0.1 | 2 | 0.9 | 0.3 | 0 | 0 | 0 | 0 | 0.2 | 0 | 51.8 | 18.6 | 5.3 | 2.3 | 0 | 3 | 4.2 | |
| DNCTraining samples=10k2016.12 | 0 | 0.4 | 1.8 | 0 | 0.8 | 0 | 0.6 | 0.3 | 0.2 | 0.2 | 0 | 0 | 0 | 0.4 | 0 | 55.1 | 12 | 0.8 | 3.9 | 0 | 2 | 3.8 | |
| DMN+Training samples=10k2016.12 | 0 | 0.3 | 1.1 | 0 | 0.5 | 0 | 2.4 | 0 | 0 | 0 | 0 | 0.2 | 0 | 0.2 | 0 | 45.3 | 4.2 | 2.1 | 0 | 0 | 1 | 2.8 | |
| EntNetTraining samples=10k, embedding dimension=100, memory slots=202016.12 | 0 | 0.1 | 4.1 | 0 | 0.3 | 0.2 | 0 | 0.5 | 0.1 | 0.6 | 0.3 | 0 | 1.3 | 0 | 0 | 0.2 | 0.5 | 0.3 | 2.3 | 0 | 0 | 0.5 | |
| MemNN2018.01 | 0 | 0 | 0 | 0 | 2 | 0 | 15 | 9 | 0 | 2 | 0 | 0 | 0 | 1 | 0 | 0 | 35 | 5 | 64 | 0 | 4 | 6.7 | |
| MemN2N2018.01 | 0 | 0.3 | 9.3 | 0 | 0.6 | 0 | 3.7 | 0.8 | 0.8 | 2.4 | 0 | 0 | 0 | 0 | 0 | 0.4 | 40.7 | 6.7 | 66.5 | 0 | 4 | 6.6 | |
| GMemN2N2018.01 | 0 | 0 | 4.5 | 0 | 0.2 | 0 | 1.8 | 0.3 | 0 | 0.2 | 0 | 0 | 0 | 0 | 0 | 0 | 27.8 | 8.5 | 31 | 0 | 3 | 3.7 | |
| DMN2018.01 | 0 | 1.8 | 4.8 | 0 | 0.7 | 0 | 3.1 | 3.5 | 0 | 2.5 | 0.1 | 0 | 0.2 | 0 | 0 | 0.6 | 40.4 | 4.7 | 65.5 | 0 | 2 | 6.4 | |
| DMN+2018.01 | 0 | 0.3 | 1.1 | 0 | 0.5 | 0 | 2.4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 45.3 | 4.2 | 2.1 | 0 | 0 | 1 | 2.8 | |
| DNC2018.01 | 0 | 0.4 | 1.8 | 0 | 0.8 | 0 | 0.6 | 0.3 | 0.2 | 0.2 | 0 | 0 | 0.1 | 0.4 | 0 | 33.1 | 12 | 0.8 | 3.9 | 0 | 2 | 2.7 | |
| RNImplementation=Authors' implementation2018.01 | 0 | 8.3 | 17.1 | 0 | 0.7 | 0 | 0.4 | 0.3 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | 4.9 | 1.6 | 2.1 | 3.2 | 0 | 2 | 2 | |
| RMN2018.01 | 0 | 0.5 | 14.7 | 0 | 0.4 | 0 | 0.5 | 0.3 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 | 0.9 | 0.3 | 2.3 | 2.9 | 0 | 1 | 1.2 | |
| EntNetTraining=jointly trained on all tasks2018.01 | 0.1 | 2.8 | 10.6 | 0 | 0.4 | 0.3 | 0.8 | 0.1 | 0 | 0 | 0 | 0 | 0 | 3.6 | 0 | 52.1 | 11.7 | 2.1 | 63 | 0 | 4 | 7.4 | |
| D-NTMTraining samples=10k2016.12 | 4.4 | 27.5 | 71.3 | 0 | 1.7 | 1.5 | 6 | 1.7 | 0.6 | 19.8 | 0 | 6.2 | 7.5 | 17.5 | 0 | 49.6 | 1.2 | 0.2 | 39.5 | 0 | 9 | 12.8 | |
| NTMTraining samples=10k2016.12 | 31.5 | 54.5 | 43.9 | 0 | 0.8 | 17.1 | 17.8 | 13.8 | 16.4 | 16.6 | 15.2 | 8.9 | 7.4 | 24.2 | 47 | 53.6 | 25.5 | 2.2 | 4.3 | 1.5 | 16 | 20.1 |