List your company
ResearchFormerly WizWand

Language Modeling on WikiText-2, C4, and PTB Average

26.68Average PerplexityShortened LLaMA + Ghost Layer
22.221652.315882.41112.5042May 15, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Shortened LLaMA + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=11-layer2026.0526.68
Shortened LLaMA + Ghost LayerModel=DeepSeek-R1-Distill-LLaMA-8B, Pruning depth (Lp)=11-layer2026.0539.8
LLM-Streamline + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=11-layer2026.0576.76
ShortGPT + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=11-layer2026.0587.61
LLM-Streamline + Ghost LayerModel=DeepSeek-R1-Distill-LLaMA-8B, Pruning depth (Lp)=11-layer2026.05124
ShortGPT + Ghost LayerModel=DeepSeek-R1-Distill-LLaMA-8B, Pruning depth (Lp)=11-layer2026.05138.14