Language Modeling on WT (val)
44.64PerplexityLSS
Evaluation results
| Method | Links | |
|---|---|---|
| LSSVariant=mHC-lite, Model Size=L2026.06 | 44.64 | |
| LSS + IdentityVariant=mHC-lite, Model Size=L2026.06 | 44.7 | |
| LSSVariant=mHC, Model Size=L2026.06 | 45.73 | |
| LSS + IdentityVariant=mHC, Model Size=L2026.06 | 45.87 | |
| BaselineVariant=mHC-lite, Model Size=L2026.06 | 46.03 | |
| IdentityVariant=mHC, Model Size=L2026.06 | 46.07 | |
| IdentityVariant=mHC-lite, Model Size=L2026.06 | 46.3 | |
| BaselineVariant=mHC, Model Size=L2026.06 | 46.35 | |
| LSSVariant=mHC-lite, Model Size=M2026.06 | 49.41 | |
| LSSVariant=mHC, Model Size=M2026.06 | 51.43 | |
| LSS + IdentityVariant=mHC, Model Size=M2026.06 | 51.54 | |
| IdentityVariant=mHC-lite, Model Size=M2026.06 | 51.86 | |
| LSS + IdentityVariant=mHC-lite, Model Size=M2026.06 | 51.96 | |
| BaselineVariant=mHC-lite, Model Size=M2026.06 | 52.74 | |
| IdentityVariant=mHC, Model Size=M2026.06 | 53.15 | |
| BaselineVariant=mHC, Model Size=M2026.06 | 53.29 |