Language Modeling on BLOOM-1b7 (train)
15.1Training PPLSurgical Reinitialization (Pass 2)
Evaluation results
| Method | Links | |
|---|---|---|
| Surgical Reinitialization (Pass 2)Surgical Stage=Pass 2, Epochs=E1, Params trained=40M (2.3%)2026.03 | 15.1 | |
| Surgical Reinitialization (Pass 1)Surgical Stage=Pass 1, Epochs=E3, Params trained=302M (17.5%)2026.03 | 15.13 | |
| BLOOM-1b7 (Stock)Surgical Stage=Stock, Params trained=—2026.03 | 16.99 |