Reading Comprehension on LAMBADA (test)
66.51AccuracyMegatron-LM
Evaluation results
| Method | Links | |
|---|---|---|
| Megatron-LMParameter Count=8.3B, Zero-shot=true2019.09 | 66.51 | |
| GPT-2Zero-shot=true2019.09 | 63.24 | |
| Megatron-LMParameter Count=2.5B, Zero-shot=true2019.09 | 61.73 | |
| Universal Transformerhalting=dynamic2018.07 | 56.25 | |
| GA w/ C-GRUArchitecture=GA, Recurrent Layer=C-GRU2018.04 | 55.69 | |
| Dhingra et al.2018.07 | 55.69 | |
| GA w/ GRUArchitecture=GA, Recurrent Layer=GRU2018.04 | 53.98 | |
| GA w/ GRU + 1-hotArchitecture=GA, Recurrent Layer=GRU, Position Feature=1-hot2018.04 | 53.38 | |
| Universal Transformersteps=6, halting=fixed2018.07 | 52.16 | |
| Chu et al. (2017)Architecture=GA2018.04 | 49 | |
| Megatron-LMParameter Count=355M, Zero-shot=true2019.09 | 45.18 | |
| Transformer2018.07 | 39.88 | |
| LSTM2018.07 | 20.07 |