Language Modeling on Core-Extended
17.08ScoreParcae
Evaluation results
| Method | Links | |
|---|---|---|
| ParcaeNumber of Parameters=1.3B, T=82026.04 | 17.08 | |
| TransformerNumber of Parameters=1.3B, T=-2026.04 | 15.9 | |
| ParcaeNumber of Parameters=770M, T=82026.04 | 15.19 | |
| TransformerNumber of Parameters=770M, T=-2026.04 | 14.2 | |
| ParcaeNumber of Parameters=370M, T=82026.04 | 12.75 | |
| TransformerNumber of Parameters=370M, T=-2026.04 | 11.71 | |
| ParcaeNumber of Parameters=140M, T=82026.04 | 9.67 | |
| TransformerNumber of Parameters=140M, T=-2026.04 | 8.8 |