Layer Pruning on GPT-2-Medium standardized evaluator
109.1Delta (%)ShortGPT (BI)
Evaluation results
| Method | Links | |
|---|---|---|
| ShortGPT (BI)n=5, Layers removed={6, 8, 10, 12, 14}2026.05 | 109.1 | |
| ShortGPT (BI)n=4, Layers removed={6, 8, 10, 12}2026.05 | 68.8 | |
| interchange-guided removal (Swap-KL)n=5, Layers removed={4, 14, 16, 20, 22}2026.05 | 52.4 | |
| ShortGPT (BI)n=3, Layers removed={6, 8, 10}2026.05 | 43 | |
| interchange-guided removal (Swap-KL)n=4, Layers removed={4, 14, 16, 20}2026.05 | 32.4 | |
| interchange-guided removal (Swap-KL)n=3, Layers removed={4, 14, 16}2026.05 | 21.9 | |
| ShortGPT (BI)n=2, Layers removed={6, 8}2026.05 | 19 | |
| interchange-guided removal (Swap-KL)n=2, Layers removed={4, 14}2026.05 | 12.7 | |
| interchange-guided removal (Swap-KL)n=1, Layers removed={4}2026.05 | 5.6 | |
| ShortGPT (BI)n=1, Layers removed={6}2026.05 | 5.3 |