Language Modeling on Wikitext-2 v1 (test)
7.33Perplexity (PPL)Dense
Evaluation results
| Method | Links | |
|---|---|---|
| DenseSparsity=0%2026.06 | 7.33 | |
| SliceGPTSparsity=25%2026.06 | 9.22 | |
| LoRAP + UniRankSparsity=25%2026.06 | 10.29 | |
| Shortened-llamaSparsity=25%2026.06 | 10.32 | |
| SkipGPTSparsity=25%2026.06 | 10.53 | |
| ShortGPTSparsity=25%2026.06 | 11.13 | |
| LoRAPSparsity=25%2026.06 | 12.82 | |
| LoRAP + UniRankSparsity=40%2026.06 | 14.03 | |
| SkipGPTSparsity=40%2026.06 | 14.35 | |
| SliceGPTSparsity=40%2026.06 | 14.87 | |
| LoRAPSparsity=40%2026.06 | 16.08 | |
| Shortened-llamaSparsity=40%2026.06 | 17.22 | |
| ShortGPTSparsity=40%2026.06 | 18.35 | |
| MoDSparsity=25%2026.06 | 34.21 | |
| D-LLMSparsity=25%2026.06 | 40.12 | |
| MoDSparsity=40%2026.06 | 40.42 | |
| D-LLMSparsity=40%2026.06 | 52.78 |