Machine Comprehension on CBT NE (test)
81.6AccuracyHumans
Evaluation Results
| Method | Links | |
|---|---|---|
| Humanscontext + query=true2016.06 | 81.6 | |
| HumansInput configuration=context+query2016.03 | 81.6 | |
| Humansinput=context+query2016.06 | 81.6 | |
| Humans (context + query)2016.06 | 81.6 | |
| AS Reader (+BookTest, ensemble)trained_on_large_set=true, ensemble=true2016.06 | 78.4 | |
| AS Reader (+BookTest)trained_on_large_set=true2016.06 | 76.2 | |
| FG fine-grained gateArchitecture=FG2016.11 | 74.96 | |
| GA Readervariant=GA (+feature, fix L(w))2016.06 | 74.9 | |
| GA fine-grained gateArchitecture=GA2016.11 | 74.64 | |
| GA Readervariant=GA (+feature, update L(w))2016.06 | 73.3 | |
| NSE2016.06 | 73.2 | |
| Munkhdalai & Yu (2016)2016.11 | 73.2 | |
| GA scalar gateArchitecture=GA2016.11 | 72.6 | |
| GA word char feat concatArchitecture=GA2016.11 | 72.56 | |
| GA word char concatArchitecture=GA2016.11 | 72.5 | |
| Iterative Alternating Neural Attentionensemble=true2016.06 | 72 | |
| AoA Reader2016.06 | 72 | |
| Iterative Attentive Readerensemble=true2016.06 | 72 | |
| Cui et al. (2016)2016.11 | 72 | |
| Sordoni et al. (2016) ensembleEnsemble=true2016.11 | 72 | |
| EpiReaderensemble=true2016.06 | 71.8 | |
| EpiReaderensemble=true2016.06 | 71.8 | |
| Trischler et al. (2016) ensembleEnsemble=true2016.11 | 71.8 | |
| GA Readervariant=GA (fix L(w))2016.06 | 71.4 | |
| AS ReaderModel variant=greedy ensemble2016.03 | 71 | |
| AS Readerensemble=true2016.06 | 71 | |
| Kadlec et al. (2016) ensembleEnsemble=true2016.11 | 71 | |
| AS Readerensemble=true2016.06 | 70.6 | |
| AS ReaderModel variant=avg ensemble2016.03 | 70.6 | |
| AS Readerensemble=true2016.06 | 70.6 | |
| GA Readervariant=GA (update L(w))2016.06 | 70.1 | |
| EpiReader2016.06 | 69.7 | |
| EpiReader2016.06 | 69.7 | |
| Trischler et al. (2016)2016.11 | 69.7 | |
| EpiReader Extractor2016.06 | 69.4 | |
| GA Readervariant=GA--2016.06 | 69 | |
| AS Reader2016.06 | 68.6 | |
| AS ReaderModel variant=single model2016.03 | 68.6 | |
| AS Reader2016.06 | 68.6 | |
| Iterative Alternating Neural Attention2016.06 | 68.6 | |
| AS Reader2016.06 | 68.6 | |
| Iterative Attentive Reader2016.06 | 68.6 | |
| Sordoni et al. (2016)2016.11 | 68.6 | |
| AS ReaderModel variant=avg for top 20%2016.03 | 68.4 | |
| MemNNs2016.06 | 66.6 | |
| MemNNsModel variant=window memory + self-sup.2016.03 | 66.6 | |
| MemNNsconfig=window memory + self-sup.2016.06 | 66.6 | |
| MemNets2016.06 | 66.6 | |
| Iterative Alternating Neural Attentionattention=fixed query attention2016.06 | 66 | |
| HumansInput configuration=query2016.03 | 52 | |
| Humansinput=query2016.06 | 52 | |
| Humans (query)2016.06 | 52 | |
| LSTMscontext + query=true2016.06 | 41.8 | |
| LSTMsInput configuration=context+query2016.03 | 41.8 | |
| LSTMsinput=context+query2016.06 | 41.8 | |
| LSTMS (context + query)2016.06 | 41.8 |