Language Modeling on WikiText-2 10K-word evaluator standardized
11.9PPL Delta (%)BI-guided
Evaluation results
| Method | Links | |
|---|---|---|
| BI-guidedn (number of layers removed)=1, Backbone=Pythia-1.4B2026.05 | 11.9 | |
| SLEB-one-shotn (number of layers removed)=1, Backbone=Pythia-1.4B2026.05 | 13.6 | |
| SLEB-iterativen (number of layers removed)=1, Backbone=Pythia-1.4B2026.05 | 13.6 | |
| Taylor importancen (number of layers removed)=1, Backbone=Pythia-1.4B2026.05 | 13.6 | |
| CKA-guidedn (number of layers removed)=1, Backbone=Pythia-1.4B2026.05 | 18.1 | |
| SLEB-one-shotn (number of layers removed)=2, Backbone=Pythia-1.4B2026.05 | 31.5 | |
| SLEB-iterativen (number of layers removed)=2, Backbone=Pythia-1.4B2026.05 | 31.5 | |
| BI-guidedn (number of layers removed)=2, Backbone=Pythia-1.4B2026.05 | 33.2 | |
| CKA-guidedn (number of layers removed)=2, Backbone=Pythia-1.4B2026.05 | 50.9 | |
| Taylor importancen (number of layers removed)=2, Backbone=Pythia-1.4B2026.05 | 56.3 | |
| Randomn (number of layers removed)=1, Backbone=Pythia-1.4B2026.05 | 74.1 | |
| BI-guidedn (number of layers removed)=3, Backbone=Pythia-1.4B2026.05 | 74.3 | |
| SLEB-iterativen (number of layers removed)=3, Backbone=Pythia-1.4B2026.05 | 79.3 | |
| CKA-guidedn (number of layers removed)=3, Backbone=Pythia-1.4B2026.05 | 88 | |
| SLEB-one-shotn (number of layers removed)=3, Backbone=Pythia-1.4B2026.05 | 116.5 | |
| Randomn (number of layers removed)=2, Backbone=Pythia-1.4B2026.05 | 152.8 | |
| Taylor importancen (number of layers removed)=3, Backbone=Pythia-1.4B2026.05 | 199.2 | |
| Randomn (number of layers removed)=3, Backbone=Pythia-1.4B2026.05 | 206.3 |