List your company
ResearchFormerly WizWand

Masked Language Modeling on C4

1.787Log PerplexityPrimer-EZ Decoder
1.784961.798731.81251.82627Sep 17, 2021
Updated 8mo ago

Evaluation results

MethodLinks
Primer-EZ DecoderParams=224M2021.091.787
Transformer++Params=224M2021.091.792
Vanilla TransformerParams=223M2021.091.838
Transformer+GeLUParams=223M2021.091.838