Masked Language Modeling on C4
1.787Log PerplexityPrimer-EZ Decoder
Evaluation Results
| Method | Links | |
|---|---|---|
| Primer-EZ DecoderParams=224M2021.09 | 1.787 | |
| Transformer++Params=224M2021.09 | 1.792 | |
| Vanilla TransformerParams=223M2021.09 | 1.838 | |
| Transformer+GeLUParams=223M2021.09 | 1.838 |