Language Modeling on Wiki
2Perplexity (PPL)Wanda
Evaluation Results
| Method | Links | |
|---|---|---|
| WandaWall-clock (h)=0.159, #GPUs=1, Model=LLaMA-2-7B, Sparsity=90%2025.10 | 2 | |
| DenseSparsity=0%, Backbone=LLaMA-2-7B2025.10 | 5.47 | |
| SAFEModel=LLaMA-2-13B, Sparsity=50%2025.10 | 5.73 | |
| ALPSModel=LLaMA-2-13B, Sparsity=50%2025.10 | 5.9 | |
| L-ADMMModel=LLaMA-2-13B, Sparsity=50%2025.10 | 5.92 | |
| WandaModel=LLaMA-2-13B, Sparsity=50%2025.10 | 5.97 | |
| SparseGPTModel=LLaMA-2-13B, Sparsity=50%2025.10 | 6.03 | |
| ELSAModel=LLaMA-2-13B, Sparsity=50%2025.10 | 6.54 | |
| SAFEModel=LLaMA-2-7B, Sparsity=50%2025.10 | 6.72 | |
| L-ADMMModel=LLaMA-2-7B, Sparsity=50%2025.10 | 6.8 | |
| MagnitudeModel=LLaMA-2-13B, Sparsity=50%2025.10 | 6.83 | |
| ALPSModel=LLaMA-2-7B, Sparsity=50%2025.10 | 6.86 | |
| SAFEModel=LLaMA-2-13B, Sparsity=60%2025.10 | 6.9 | |
| WandaModel=LLaMA-2-7B, Sparsity=50%2025.10 | 6.92 | |
| SparseGPTModel=LLaMA-2-7B, Sparsity=50%2025.10 | 7.01 | |
| DenseModel=Llama-3.1-8B, Calibration Method=Baseline2026.03 | 7.03 | |
| AWQ (W4A16)Model=Llama-3.1-8B, Calibration Method=COLA2026.03 | 7.21 | |
| SparseLLMModel=LLaMA-2-7B, Sparsity=50%2025.10 | 7.23 | |
| AWQ (W4A16)Model=Llama-3.1-8B, Calibration Method=ZipCal2026.03 | 7.31 | |
| AWQ (W4A16)Model=Llama-3.1-8B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 7.32 | |
| ELSAModel=LLaMA-2-7B, Sparsity=50%2025.10 | 7.5 | |
| Wanda (25%)Model=Llama-3.1-8B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 7.52 | |
| ALPSModel=LLaMA-2-13B, Sparsity=60%2025.10 | 7.56 | |
| L-ADMMModel=LLaMA-2-13B, Sparsity=60%2025.10 | 7.57 | |
| Wanda (25%)Model=Llama-3.1-8B, Calibration Method=ZipCal2026.03 | 7.6 | |
| Wanda (25%)Model=Llama-3.1-8B, Calibration Method=COLA2026.03 | 7.62 | |
| DenseModel=Gemma-2-9B, Calibration Method=Baseline2026.03 | 7.63 | |
| GPTQ (W4A16)Model=Llama-3.1-8B, Calibration Method=COLA2026.03 | 7.71 | |
| GPTQ (W4A16)Model=Llama-3.1-8B, Calibration Method=ZipCal2026.03 | 7.71 | |
| Wanda (25%)Model=Gemma-2-9B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 7.86 | |
| Wanda (25%)Model=Gemma-2-9B, Calibration Method=COLA2026.03 | 7.87 | |
| Wanda (25%)Model=Gemma-2-9B, Calibration Method=ZipCal2026.03 | 7.92 | |
| GPTQ (W4A16)Model=Gemma-2-9B, Calibration Method=COLA2026.03 | 7.93 | |
| GPTQ (W4A16)Model=Gemma-2-9B, Calibration Method=ZipCal2026.03 | 7.93 | |
| ELSAModel=LLaMA-2-13B, Sparsity=60%2025.10 | 7.96 | |
| SAFESparsity=4:8, Backbone=LLaMA-2-7B2025.10 | 8.043 | |
| ALPSSparsity=4:8, Backbone=LLaMA-2-7B2025.10 | 8.103 | |
| L-ADMMSparsity=4:8, Backbone=LLaMA-2-7B2025.10 | 8.12 | |
| SparseGPTModel=LLaMA-2-13B, Sparsity=60%2025.10 | 8.27 | |
| WandaModel=LLaMA-2-13B, Sparsity=60%2025.10 | 8.4 | |
| GPTQ (W4A16)Model=Gemma-2-9B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 8.44 | |
| SparseGPTSparsity=4:8, Backbone=LLaMA-2-7B2025.10 | 8.508 | |
| FP16Model=Qwen-2.5-3B-Instruct2026.04 | 8.56 | |
| GPTQ (W4A16)Model=Llama-3.1-8B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 8.59 | |
| WandaSparsity=4:8, Backbone=LLaMA-2-7B2025.10 | 8.603 | |
| SparseLLMSparsity=4:8, Backbone=LLaMA-2-7B2025.10 | 8.679 | |
| SAFEModel=LLaMA-2-7B, Sparsity=60%2025.10 | 9.02 | |
| ELSAModel=LLaMA-2-7B, Sparsity=60%2025.10 | 9.16 | |
| ELSASparsity=4:8, Backbone=LLaMA-2-7B2025.10 | 9.2 | |
| ALPSModel=LLaMA-2-7B, Sparsity=60%2025.10 | 9.33 | |
| L-ADMMModel=LLaMA-2-7B, Sparsity=60%2025.10 | 9.4 | |
| SAFESparsity=2:4, Backbone=LLaMA-2-7B2025.10 | 9.914 | |
| ALPSSparsity=2:4, Backbone=LLaMA-2-7B2025.10 | 9.945 | |
| ELSASparsity=2:4, Backbone=LLaMA-2-7B2025.10 | 10.15 | |
| L-ADMMSparsity=2:4, Backbone=LLaMA-2-7B2025.10 | 10.19 | |
| SparseGPTModel=LLaMA-2-7B, Sparsity=60%2025.10 | 10.2 | |
| SAFEModel=LLaMA-3.2-3B, Sparsity=50%2025.10 | 10.68 | |
| SparseLLMModel=LLaMA-2-7B, Sparsity=60%2025.10 | 10.74 | |
| WandaModel=LLaMA-2-7B, Sparsity=60%2025.10 | 10.79 | |
| SparseGPTSparsity=2:4, Backbone=LLaMA-2-7B2025.10 | 10.87 | |
| ALPSModel=Gemma-2-2B, Sparsity=50%2025.10 | 10.93 | |
| L-ADMMModel=Gemma-2-2B, Sparsity=50%2025.10 | 11.02 | |
| ELSAModel=LLaMA-2-13B, Sparsity=70%2025.10 | 11.14 | |
| SparseLLMSparsity=2:4, Backbone=LLaMA-2-7B2025.10 | 11.29 | |
| ALPSModel=LLaMA-3.2-3B, Sparsity=50%2025.10 | 11.31 | |
| L-ADMMModel=LLaMA-3.2-3B, Sparsity=50%2025.10 | 11.56 | |
| SparseGPTModel=Gemma-2-2B, Sparsity=50%2025.10 | 11.58 | |
| SAFEModel=Gemma-2-2B, Sparsity=50%2025.10 | 11.61 | |
| MeSHModel Backbone=Pythia-6.9B, Scheme=Recursive (-31.25%), Layers=6+10R2+6, Variant=+mesh2025.10 | 11.64 | |
| 2SSP (25%)Model=Gemma-2-9B, Calibration Method=ZipCal2026.03 | 11.68 | |
| MagnitudeModel=LLaMA-2-13B, Sparsity=60%2025.10 | 11.82 | |
| ELSAModel=LLaMA-3.2-3B, Sparsity=50%2025.10 | 12.06 | |
| WandaModel=Gemma-2-2B, Sparsity=50%2025.10 | 12.07 | |
| WandaSparsity=2:4, Backbone=LLaMA-2-7B2025.10 | 12.13 | |
| Recursive baselineModel Backbone=Pythia-6.9B, Scheme=Recursive (-31.25%), Layers=6+10R2+6, Variant=base2025.10 | 12.14 | |
| SparseGPTModel=LLaMA-3.2-3B, Sparsity=50%2025.10 | 12.27 | |
| SAFEModel=LLaMA-2-13B, Sparsity=70%2025.10 | 12.47 | |
| 2SSP (25%)Model=Gemma-2-9B, Calibration Method=COLA2026.03 | 12.56 | |
| WandaModel=LLaMA-3.2-3B, Sparsity=50%2025.10 | 13.01 | |
| ELSAModel=Gemma-2-2B, Sparsity=50%2025.10 | 13.05 | |
| ELSAModel=LLaMA-2-7B, Sparsity=70%2025.10 | 13.2 | |
| MeSHModel Backbone=Pythia-2.8B, Scheme=Recursive (-31.25%), Layers=6+10R2+6, Variant=+mesh2025.10 | 13.6 | |
| 2SSP (25%)Model=Gemma-2-9B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 14.14 | |
| ALPSModel=LLaMA-2-13B, Sparsity=70%2025.10 | 14.17 | |
| Recursive baselineModel Backbone=Pythia-2.8B, Scheme=Recursive (-31.25%), Layers=6+10R2+6, Variant=base2025.10 | 14.18 | |
| ALPSModel=Gemma-2-2B, Sparsity=60%2025.10 | 14.42 | |
| L-ADMMModel=Gemma-2-2B, Sparsity=60%2025.10 | 14.65 | |
| L-ADMMModel=LLaMA-2-13B, Sparsity=70%2025.10 | 14.81 | |
| SAFEModel=Gemma-2-2B, Sparsity=60%2025.10 | 15.22 | |
| MeSHModel Size=Pythia-1.4B, Scheme=Recursive, Layers=4+8R2+4, Variant=+mesh, Parameter reduction percentage=-33.3%2025.10 | 15.84 | |
| MagnitudeSparsity=4:8, Backbone=LLaMA-2-7B2025.10 | 15.91 | |
| ELSAModel=Gemma-2-2B, Sparsity=60%2025.10 | 15.93 | |
| VanillaModel Size=Pythia-1.4B, Scheme=Vanilla, Layers=242025.10 | 15.97 | |
| AWQ (W4A16)Model=Gemma-2-9B, Calibration Method=COLA2026.03 | 16.01 | |
| AWQ (W4A16)Model=Gemma-2-9B, Calibration Method=ZipCal2026.03 | 16.01 | |
| AWQ (W4A16)Model=Gemma-2-9B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 16.01 | |
| MagnitudeModel=LLaMA-2-7B, Sparsity=50%2025.10 | 16.03 | |
| SAFEModel=OPT-1.3B, Sparsity=50%2025.10 | 16.38 | |
| SparseGPTModel=Gemma-2-2B, Sparsity=60%2025.10 | 16.53 | |
| ELSAModel=LLaMA-3.2-3B, Sparsity=60%2025.10 | 16.65 |