Language Modeling on FineWeb-Edu 1.4B tokens (val)
3.271LossDeVA_S8
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeVA_S8Model Architecture=NanoGPT, Total Parameters=274M, Training Corpus Size=1.4B tokens2026.02 | 3.271 | 16.49 | |
| SOAPModel Architecture=NanoGPT, Total Parameters=274M, Training Corpus Size=1.4B tokens2026.02 | 3.304 | 15.79 | |
| MuonModel Architecture=NanoGPT, Total Parameters=274M, Training Corpus Size=1.4B tokens2026.02 | 3.305 | 10.72 |