Language Modeling on WikiText-2, C4, and PTB Average
26.68Average PerplexityShortened LLaMA + Ghost Layer
Evaluation Results
| Method | Links | |
|---|---|---|
| Shortened LLaMA + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=11-layer2026.05 | 26.68 | |
| Shortened LLaMA + Ghost LayerModel=DeepSeek-R1-Distill-LLaMA-8B, Pruning depth (Lp)=11-layer2026.05 | 39.8 | |
| LLM-Streamline + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=11-layer2026.05 | 76.76 | |
| ShortGPT + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=11-layer2026.05 | 87.61 | |
| LLM-Streamline + Ghost LayerModel=DeepSeek-R1-Distill-LLaMA-8B, Pruning depth (Lp)=11-layer2026.05 | 124 | |
| ShortGPT + Ghost LayerModel=DeepSeek-R1-Distill-LLaMA-8B, Pruning depth (Lp)=11-layer2026.05 | 138.14 |