Language Modeling on Language modeling objective (test)
2.341LossSMART
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SMARTParameters (M)=45.512025.12 | 2.341 | 0.0091 | |
| GPT-2Parameters (M)=124.42025.12 | 2.787 | 0.0029 | |
| Pure TransformerParameters (M)=522025.12 | 3.456 | 0.0056 | |
| DistilBERTParameters (M)=89.82025.12 | 10.43 | 0.0011 | |
| BERTParameters (M)=1332025.12 | 10.46 | 0.0007 |