Language Modeling on LLaMA 13B 2
4.57Perplexity (PPL)Statistical Correction (Ours)
Evaluation results
| Method | Links | |
|---|---|---|
| Statistical Correction (Ours)Unstructured Sparsity Level=10%2026.02 | 4.57 | |
| MagnitudeUnstructured Sparsity Level=10%2026.02 | 4.59 | |
| SparseGPTUnstructured Sparsity Level=10%2026.02 | 4.59 | |
| WandaUnstructured Sparsity Level=10%2026.02 | 4.6 | |
| Statistical Correction (Ours)Unstructured Sparsity Level=30%2026.02 | 4.72 | |
| SparseGPTUnstructured Sparsity Level=30%2026.02 | 4.78 | |
| WandaUnstructured Sparsity Level=30%2026.02 | 4.79 | |
| MagnitudeUnstructured Sparsity Level=30%2026.02 | 4.82 | |
| Statistical Correction (Ours)Unstructured Sparsity Level=50%2026.02 | 5.32 | |
| WandaUnstructured Sparsity Level=50%2026.02 | 5.59 | |
| SparseGPTUnstructured Sparsity Level=50%2026.02 | 5.63 | |
| Statistical Correction (Ours)Unstructured Sparsity Level=60%2026.02 | 6.33 | |
| MagnitudeUnstructured Sparsity Level=50%2026.02 | 6.37 | |
| WandaUnstructured Sparsity Level=60%2026.02 | 7.02 | |
| SparseGPTUnstructured Sparsity Level=60%2026.02 | 7.74 | |
| Statistical Correction (Ours)Unstructured Sparsity Level=70%2026.02 | 10.52 | |
| MagnitudeUnstructured Sparsity Level=60%2026.02 | 11.22 | |
| SparseGPTUnstructured Sparsity Level=70%2026.02 | 18.04 | |
| WandaUnstructured Sparsity Level=70%2026.02 | 45.77 | |
| MagnitudeUnstructured Sparsity Level=70%2026.02 | 275.23 |