Attention Head Health Analysis on BLOOM-1b7
379Healthy Heads CountSurgical Reinitialization (Pass 2)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Surgical Reinitialization (Pass 2)Surgical Stage=Pass 2, Epochs=E1, Params trained=40M (2.3%)2026.03 | 379 | 1 | 0 | 4 | |
| Surgical Reinitialization (Pass 1)Surgical Stage=Pass 1, Epochs=E3, Params trained=302M (17.5%)2026.03 | 341 | 39 | 0 | 4 | |
| BLOOM-1b7 (Stock)Surgical Stage=Stock, Params trained=—2026.03 | 242 | 136 | 3 | 3 |