Machine Comprehension on CBT-CN (test)
83.7AccuracyAS Reader (+BookTest, ensemble)
Evaluation Results
| Method | Links | |
|---|---|---|
| AS Reader (+BookTest, ensemble)trained_on_large_set=true, ensemble=true2016.06 | 83.7 | |
| Humanscontext + query=true2016.06 | 81.6 | |
| HumansInput configuration=context+query2016.03 | 81.6 | |
| Humansinput=context+query2016.06 | 81.6 | |
| Humans (context + query)2016.06 | 81.6 | |
| AS Reader (+BookTest)trained_on_large_set=true2016.06 | 80.8 | |
| FG fine-grained gateArchitecture=FG2016.11 | 72.04 | |
| NSE2016.06 | 71.9 | |
| Munkhdalai & Yu (2016)2016.11 | 71.9 | |
| Iterative Alternating Neural Attentionensemble=true2016.06 | 71 | |
| Iterative Attentive Readerensemble=true2016.06 | 71 | |
| Sordoni et al. (2016) ensembleEnsemble=true2016.11 | 71 | |
| GA fine-grained gateArchitecture=GA2016.11 | 70.84 | |
| GA Readervariant=GA (+feature, fix L(w))2016.06 | 70.7 | |
| EpiReaderensemble=true2016.06 | 70.6 | |
| EpiReaderensemble=true2016.06 | 70.6 | |
| Trischler et al. (2016) ensembleEnsemble=true2016.11 | 70.6 | |
| GA Readervariant=GA (+feature, update L(w))2016.06 | 69.8 | |
| GA word char concatArchitecture=GA2016.11 | 69.6 | |
| AoA Reader2016.06 | 69.4 | |
| Cui et al. (2016)2016.11 | 69.4 | |
| GA word char feat concatArchitecture=GA2016.11 | 69.28 | |
| Iterative Alternating Neural Attention2016.06 | 69.2 | |
| Iterative Attentive Reader2016.06 | 69.2 | |
| Sordoni et al. (2016)2016.11 | 69.2 | |
| GA scalar gateArchitecture=GA2016.11 | 69.08 | |
| AS Readerensemble=true2016.06 | 68.9 | |
| AS ReaderModel variant=avg ensemble2016.03 | 68.9 | |
| AS Readerensemble=true2016.06 | 68.9 | |
| AS Readerensemble=true2016.06 | 68.9 | |
| Kadlec et al. (2016) ensembleEnsemble=true2016.11 | 68.9 | |
| GA Readervariant=GA (fix L(w))2016.06 | 68 | |
| AS ReaderModel variant=greedy ensemble2016.03 | 67.5 | |
| EpiReader2016.06 | 67.4 | |
| EpiReader2016.06 | 67.4 | |
| Trischler et al. (2016)2016.11 | 67.4 | |
| GA Readervariant=GA (update L(w))2016.06 | 67.3 | |
| EpiReader Extractor2016.06 | 66.7 | |
| HumansInput configuration=query2016.03 | 64.4 | |
| Humansinput=query2016.06 | 64.4 | |
| Humans (query)2016.06 | 64.4 | |
| Iterative Alternating Neural Attentionattention=fixed query attention2016.06 | 64.3 | |
| GA Readervariant=GA--2016.06 | 63.9 | |
| AS Reader2016.06 | 63.4 | |
| AS ReaderModel variant=single model2016.03 | 63.4 | |
| AS Reader2016.06 | 63.4 | |
| AS Reader2016.06 | 63.4 | |
| AS ReaderModel variant=avg for top 20%2016.03 | 63.2 | |
| MemNNs2016.06 | 63 | |
| MemNNsModel variant=window memory + self-sup.2016.03 | 63 | |
| MemNNsconfig=window memory + self-sup.2016.06 | 63 | |
| MemNets2016.06 | 63 | |
| LSTMscontext + query=true2016.06 | 56 | |
| LSTMsInput configuration=context+query2016.03 | 56 | |
| LSTMsinput=context+query2016.06 | 56 | |
| LSTMS (context + query)2016.06 | 56 |