Language Modeling on Wiki-PPL
11.72Wiki-PPLMIDUS-HML
Evaluation results
| Method | Links | |
|---|---|---|
| MIDUS-HMLBase Model=Llama-3.2-1B, Pre-training=CPT-1B, SFT Dataset=Databricks-Dolly-15k2025.12 | 11.72 | |
| LESABase Model=Llama-3.2-1B, Pre-training=CPT-1B, SFT Dataset=Databricks-Dolly-15k2025.12 | 11.77 | |
| OpT-DeUSBase Model=Llama-3.2-1B, Pre-training=CPT-1B, SFT Dataset=Databricks-Dolly-15k2025.12 | 11.78 | |
| Avg-DeUSBase Model=Llama-3.2-1B, Pre-training=CPT-1B, SFT Dataset=Databricks-Dolly-15k2025.12 | 12.1 | |
| Llama ProBase Model=Llama-3.2-1B, Pre-training=CPT-1B, SFT Dataset=Databricks-Dolly-15k2025.12 | 12.47 | |
| BaseBase Model=Llama-3.2-1B, Pre-training=CPT-1B, SFT Dataset=Databricks-Dolly-15k2025.12 | 13.07 | |
| SOLARBase Model=Llama-3.2-1B, Pre-training=CPT-1B, SFT Dataset=Databricks-Dolly-15k2025.12 | 13.19 |