Language Modeling on Wikitext-2 (PPL)
5.47Perplexity (PPL)Dense
Evaluation results
| Method | Links | |
|---|---|---|
| DenseModel=LLaMA2-7B, Sparsity=Dense2026.06 | 5.47 | |
| RIAModel=LLaMA2-7B, Criterion=S-S, Sparsity=30%2026.06 | 5.64 | |
| DenseModel=LLaMA-7B, Sparsity=Dense2026.06 | 5.68 | |
| WandaModel=LLaMA2-7B, Criterion=S-S, Sparsity=30%2026.06 | 5.73 | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=30%2026.06 | 5.74 | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=30%2026.06 | 5.77 | |
| RIAModel=LLaMA-7B, Criterion=S-S, Sparsity=30%2026.06 | 5.86 | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=30%2026.06 | 5.95 | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=30%2026.06 | 5.96 | |
| RIAModel=LLaMA2-7B, Criterion=S-S, Sparsity=40%2026.06 | 5.96 | |
| WandaModel=LLaMA-7B, Criterion=S-S, Sparsity=30%2026.06 | 5.99 | |
| WandaModel=LLaMA2-7B, Criterion=S-S, Sparsity=40%2026.06 | 6.05 | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=40%2026.06 | 6.07 | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=40%2026.06 | 6.1 | |
| DenseModel=LLaMA3-8B, Sparsity=Dense2026.06 | 6.13 | |
| RIAModel=LLaMA-7B, Criterion=S-S, Sparsity=40%2026.06 | 6.21 | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=40%2026.06 | 6.32 | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=40%2026.06 | 6.33 | |
| WandaModel=LLaMA-7B, Criterion=S-S, Sparsity=40%2026.06 | 6.38 | |
| RIAModel=LLaMA3-8B, Criterion=S-S, Sparsity=30%2026.06 | 6.6 | |
| WandaModel=LLaMA3-8B, Criterion=S-S, Sparsity=30%2026.06 | 6.67 | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=30%2026.06 | 6.67 | |
| SparseGPTModel=LLaMA3-8B, Criterion=S-S, Sparsity=30%2026.06 | 6.69 | |
| RIAModel=LLaMA2-7B, Criterion=S-S, Sparsity=50%2026.06 | 6.8 | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=50%2026.06 | 6.86 | |
| WandaModel=LLaMA2-7B, Criterion=S-S, Sparsity=50%2026.06 | 6.92 | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=50%2026.06 | 6.99 | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=50%2026.06 | 6.99 | |
| RIAModel=LLaMA-7B, Criterion=S-S, Sparsity=50%2026.06 | 7.13 | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=50%2026.06 | 7.19 | |
| WandaModel=LLaMA-7B, Criterion=S-S, Sparsity=50%2026.06 | 7.26 | |
| RIAModel=LLaMA3-8B, Criterion=S-S, Sparsity=40%2026.06 | 7.36 | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=40%2026.06 | 7.43 | |
| WandaModel=LLaMA3-8B, Criterion=S-S, Sparsity=40%2026.06 | 7.44 | |
| SparseGPTModel=LLaMA3-8B, Criterion=S-S, Sparsity=40%2026.06 | 7.48 | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=60%2026.06 | 8.52 | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=60%2026.06 | 8.64 | |
| RIAModel=LLaMA3-8B, Criterion=S-S, Sparsity=50%2026.06 | 9.33 | |
| SparseGPTModel=LLaMA3-8B, Criterion=S-S, Sparsity=50%2026.06 | 9.42 | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=50%2026.06 | 9.48 | |
| WandaModel=LLaMA3-8B, Criterion=S-S, Sparsity=50%2026.06 | 9.82 | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=60%2026.06 | 10.13 | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=60%2026.06 | 10.18 | |
| RIAModel=LLaMA2-7B, Criterion=S-S, Sparsity=60%2026.06 | 10.4 | |
| RIAModel=LLaMA-7B, Criterion=S-S, Sparsity=60%2026.06 | 10.63 | |
| WandaModel=LLaMA-7B, Criterion=S-S, Sparsity=60%2026.06 | 10.73 | |
| WandaModel=LLaMA2-7B, Criterion=S-S, Sparsity=60%2026.06 | 10.77 | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=60%2026.06 | 12.82 | |
| VanillaBackbone=Mistral-7B-Instruct2026.05 | 13.9 | |
| DT-CD⋆Backbone=Mistral-7B-Instruct2026.05 | 13.9 | |
| COFTBackbone=Mistral-7B-Instruct2026.05 | 13.9 | |
| SDDBackbone=Mistral-7B-Instruct2026.05 | 14.1 | |
| DExpertsBackbone=Mistral-7B-Instruct2026.05 | 14.2 | |
| DenseModel=OPT-1.3B, Sparsity=Dense2026.06 | 14.62 | |
| ICPModel=OPT-1.3B, Sparsity=30%2026.06 | 14.67 | |
| WandaModel=OPT-1.3B, Criterion=S-S, Sparsity=30%2026.06 | 15 | |
| ICPModel=OPT-1.3B, Sparsity=40%2026.06 | 15.04 | |
| EPTSModel=OPT-1.3B, Criterion=M-S, Sparsity=30%2026.06 | 15.1 | |
| SparseGPTModel=OPT-1.3B, Criterion=S-S, Sparsity=30%2026.06 | 15.16 | |
| RIAModel=OPT-1.3B, Criterion=S-S, Sparsity=30%2026.06 | 15.16 | |
| VanillaBackbone=LLaMA-2-13B2026.05 | 15.3 | |
| DT-CD⋆Backbone=LLaMA-2-13B2026.05 | 15.4 | |
| COFTBackbone=LLaMA-2-13B2026.05 | 15.4 | |
| SparseGPTModel=LLaMA3-8B, Criterion=S-S, Sparsity=60%2026.06 | 15.43 | |
| SDDBackbone=LLaMA-2-13B2026.05 | 15.6 | |
| DExpertsBackbone=LLaMA-2-13B2026.05 | 15.8 | |
| WandaModel=OPT-1.3B, Criterion=S-S, Sparsity=40%2026.06 | 15.87 | |
| RIAModel=OPT-1.3B, Criterion=S-S, Sparsity=40%2026.06 | 15.91 | |
| EPTSModel=OPT-1.3B, Criterion=M-S, Sparsity=40%2026.06 | 16.03 | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=70%2026.06 | 16.47 | |
| ICPModel=OPT-1.3B, Sparsity=50%2026.06 | 16.49 | |
| SparseGPTModel=OPT-1.3B, Criterion=S-S, Sparsity=40%2026.06 | 16.5 | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=70%2026.06 | 16.94 | |
| SparseGPTModel=OPT-1.3B, Criterion=S-S, Sparsity=50%2026.06 | 17.45 | |
| EPTSModel=OPT-1.3B, Criterion=M-S, Sparsity=50%2026.06 | 17.74 | |
| RIAModel=OPT-1.3B, Criterion=S-S, Sparsity=50%2026.06 | 18.05 | |
| WandaModel=OPT-1.3B, Criterion=S-S, Sparsity=50%2026.06 | 18.42 | |
| RIAModel=LLaMA3-8B, Criterion=S-S, Sparsity=60%2026.06 | 19.64 | |
| EPTSModel=OPT-1.3B, Criterion=M-S, Sparsity=60%2026.06 | 20.94 | |
| ICPModel=OPT-1.3B, Sparsity=60%2026.06 | 21.17 | |
| DenseModel=OPT-350M, Sparsity=Dense2026.06 | 22 | |
| SparseGPTModel=OPT-1.3B, Criterion=S-S, Sparsity=60%2026.06 | 22.17 | |
| SparseGPTModel=OPT-350M, Criterion=S-S, Sparsity=30%2026.06 | 23.19 | |
| WandaModel=OPT-350M, Criterion=S-S, Sparsity=30%2026.06 | 23.49 | |
| EPTSModel=OPT-350M, Criterion=M-S, Sparsity=30%2026.06 | 23.51 | |
| WandaModel=LLaMA3-8B, Criterion=S-S, Sparsity=60%2026.06 | 23.52 | |
| RIAModel=OPT-350M, Criterion=S-S, Sparsity=30%2026.06 | 23.54 | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=70%2026.06 | 25.02 | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=70%2026.06 | 25.78 | |
| SparseGPTModel=OPT-350M, Criterion=S-S, Sparsity=40%2026.06 | 25.8 | |
| RIAModel=OPT-1.3B, Criterion=S-S, Sparsity=60%2026.06 | 26.28 | |
| WandaModel=OPT-1.3B, Criterion=S-S, Sparsity=60%2026.06 | 26.49 | |
| WandaModel=OPT-350M, Criterion=S-S, Sparsity=40%2026.06 | 26.54 | |
| RIAModel=OPT-350M, Criterion=S-S, Sparsity=40%2026.06 | 26.55 | |
| EPTSModel=OPT-350M, Criterion=M-S, Sparsity=40%2026.06 | 26.65 | |
| DenseModel=OPT-125M, Sparsity=Dense2026.06 | 27.65 | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=70%2026.06 | 28.06 | |
| WandaModel=OPT-125M, Criterion=S-S, Sparsity=30%2026.06 | 28.1 | |
| EPTSModel=OPT-125M, Criterion=M-S, Sparsity=30%2026.06 | 28.37 | |
| RIAModel=OPT-125M, Criterion=S-S, Sparsity=30%2026.06 | 28.63 |