Language Modeling on Dolma
218.8PerplexitysHC
Evaluation results
| Method | Links | |
|---|---|---|
| sHCModel Scale=L, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 218.8 | |
| mHC-liteModel Scale=L, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 223 | |
| mHCModel Scale=L, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 228.4 | |
| RCModel Scale=L, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 260.7 | |
| HCModel Scale=L, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 264.3 | |
| mHC-liteModel Scale=M, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 344 | |
| mHCModel Scale=M, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 351.8 | |
| sHCModel Scale=M, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 358.8 | |
| HCModel Scale=M, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 367.4 | |
| RCModel Scale=M, Zero-shot evaluation=true, Training Dataset=FineWeb-Edu2026.03 | 397 |