Language Modeling on WikiText2 2016 (test)
3.32PerplexityDense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseModel=LLaMA2-70B, Sparsity=0%2025.12 | 3.32 | |
| SoBPModel=LLaMA2-70B, Sparsity=20%2025.12 | 3.88 | |
| OBCModel=LLaMA2-70B, Sparsity=20%2025.12 | 3.95 | |
| O-FANGModel=LLaMA2-70B, Sparsity=20%2025.12 | 3.97 | |
| F-FANGModel=LLaMA2-70B, Sparsity=20%2025.12 | 3.98 | |
| FLAPModel=LLaMA2-70B, Sparsity=20%2025.12 | 4.12 | |
| SoBPModel=LLaMA2-70B, Sparsity=30%2025.12 | 4.36 | |
| OBCModel=LLaMA2-70B, Sparsity=30%2025.12 | 4.4 | |
| SliceGPTModel=LLaMA2-70B, Sparsity=20%2025.12 | 4.46 | |
| F-FANGModel=LLaMA2-70B, Sparsity=30%2025.12 | 4.46 | |
| O-FANGModel=LLaMA2-70B, Sparsity=30%2025.12 | 4.46 | |
| SVD-LLMModel=LLaMA2-70B, Sparsity=20%2025.12 | 4.66 | |
| FLAPModel=LLaMA2-70B, Sparsity=30%2025.12 | 4.82 | |
| OBCModel=LLaMA2-70B, Sparsity=40%2025.12 | 4.87 | |
| DenseModel=LLaMA2-13B, Sparsity=0%2025.12 | 4.88 | |
| SoBPModel=LLaMA2-70B, Sparsity=40%2025.12 | 4.96 | |
| O-FANGModel=LLaMA2-70B, Sparsity=40%2025.12 | 5.01 | |
| F-FANGModel=LLaMA2-70B, Sparsity=40%2025.12 | 5.06 | |
| SliceGPTModel=LLaMA2-70B, Sparsity=30%2025.12 | 5.41 | |
| SVD-LLMModel=LLaMA2-70B, Sparsity=30%2025.12 | 5.44 | |
| DenseModel=LLaMA2-7B, Sparsity=0%2025.12 | 5.47 | |
| OBCModel=LLaMA2-13B, Sparsity=20%2025.12 | 5.53 | |
| O-FANGModel=LLaMA2-13B, Sparsity=20%2025.12 | 5.57 | |
| SoBPModel=LLaMA2-13B, Sparsity=20%2025.12 | 5.62 | |
| DenseModel=LLaMA1-7B, Sparsity=0%2025.12 | 5.68 | |
| F-FANGModel=LLaMA2-13B, Sparsity=20%2025.12 | 5.82 | |
| SliceGPTModel=LLaMA2-13B, Sparsity=20%2025.12 | 6.06 | |
| O-FANGModel=LLaMA2-13B, Sparsity=30%2025.12 | 6.17 | |
| FLAPModel=LLaMA2-70B, Sparsity=40%2025.12 | 6.24 | |
| SoBPModel=LLaMA2-13B, Sparsity=30%2025.12 | 6.27 | |
| OBCModel=LLaMA2-7B, Sparsity=20%2025.12 | 6.3 | |
| O-FANGModel=LLaMA2-7B, Sparsity=20%2025.12 | 6.31 | |
| FLAPModel=LLaMA2-13B, Sparsity=20%2025.12 | 6.31 | |
| O-FANGModel=LLaMA1-7B, Sparsity=20%2025.12 | 6.43 | |
| SoBPModel=LLaMA2-7B, Sparsity=20%2025.12 | 6.53 | |
| F-FANGModel=LLaMA2-13B, Sparsity=30%2025.12 | 6.53 | |
| OBCModel=LLaMA1-7B, Sparsity=20%2025.12 | 6.61 | |
| F-FANGModel=LLaMA2-7B, Sparsity=20%2025.12 | 6.63 | |
| SVD-LLMModel=LLaMA2-13B, Sparsity=20%2025.12 | 6.66 | |
| F-FANGModel=LLaMA1-7B, Sparsity=20%2025.12 | 6.7 | |
| SoBPModel=LLaMA1-7B, Sparsity=20%2025.12 | 6.78 | |
| SVD-LLMModel=LLaMA2-70B, Sparsity=40%2025.12 | 6.83 | |
| SliceGPTModel=LLaMA2-7B, Sparsity=20%2025.12 | 6.84 | |
| FLAPModel=LLaMA1-7B, Sparsity=20%2025.12 | 6.89 | |
| SliceGPTModel=LLaMA1-7B, Sparsity=20%2025.12 | 6.99 | |
| O-FANGModel=LLaMA2-13B, Sparsity=40%2025.12 | 7.04 | |
| SliceGPTModel=LLaMA2-70B, Sparsity=40%2025.12 | 7.08 | |
| FLAPModel=LLaMA2-7B, Sparsity=20%2025.12 | 7.16 | |
| O-FANGModel=LLaMA1-7B, Sparsity=30%2025.12 | 7.23 | |
| O-FANGModel=LLaMA2-7B, Sparsity=30%2025.12 | 7.23 | |
| OBCModel=LLaMA2-7B, Sparsity=30%2025.12 | 7.34 | |
| OBCModel=LLaMA1-7B, Sparsity=30%2025.12 | 7.38 | |
| SoBPModel=LLaMA2-13B, Sparsity=40%2025.12 | 7.39 | |
| SliceGPTModel=LLaMA2-13B, Sparsity=30%2025.12 | 7.44 | |
| SoBPModel=LLaMA1-7B, Sparsity=30%2025.12 | 7.57 | |
| FLAPModel=LLaMA2-13B, Sparsity=30%2025.12 | 7.57 | |
| SoBPModel=LLaMA2-7B, Sparsity=30%2025.12 | 7.58 | |
| OBCModel=LLaMA2-13B, Sparsity=30%2025.12 | 7.72 | |
| F-FANGModel=LLaMA2-7B, Sparsity=30%2025.12 | 7.78 | |
| F-FANGModel=LLaMA2-13B, Sparsity=40%2025.12 | 7.84 | |
| SVD-LLMModel=LLaMA1-7B, Sparsity=20%2025.12 | 7.89 | |
| F-FANGModel=LLaMA1-7B, Sparsity=30%2025.12 | 7.94 | |
| SVD-LLMModel=LLaMA2-13B, Sparsity=30%2025.12 | 8 | |
| FLAPModel=LLaMA1-7B, Sparsity=30%2025.12 | 8.23 | |
| SVD-LLMModel=LLaMA2-7B, Sparsity=20%2025.12 | 8.38 | |
| O-FANGModel=LLaMA1-7B, Sparsity=40%2025.12 | 8.56 | |
| SliceGPTModel=LLaMA2-7B, Sparsity=30%2025.12 | 8.64 | |
| O-FANGModel=LLaMA2-7B, Sparsity=40%2025.12 | 8.67 | |
| SliceGPTModel=LLaMA1-7B, Sparsity=30%2025.12 | 8.69 | |
| FLAPModel=LLaMA2-7B, Sparsity=30%2025.12 | 8.85 | |
| OBCModel=LLaMA1-7B, Sparsity=40%2025.12 | 8.85 | |
| FLAPModel=LLaMA2-13B, Sparsity=40%2025.12 | 9.07 | |
| SoBPModel=LLaMA1-7B, Sparsity=40%2025.12 | 9.09 | |
| OBCModel=LLaMA2-7B, Sparsity=40%2025.12 | 9.13 | |
| SoBPModel=LLaMA2-7B, Sparsity=40%2025.12 | 9.28 | |
| SVD-LLMModel=LLaMA1-7B, Sparsity=30%2025.12 | 9.52 | |
| FLAPModel=LLaMA1-7B, Sparsity=40%2025.12 | 10.16 | |
| F-FANGModel=LLaMA1-7B, Sparsity=40%2025.12 | 10.31 | |
| F-FANGModel=LLaMA2-7B, Sparsity=40%2025.12 | 10.34 | |
| SliceGPTModel=LLaMA2-13B, Sparsity=40%2025.12 | 10.6 | |
| SVD-LLMModel=LLaMA2-7B, Sparsity=30%2025.12 | 10.66 | |
| SVD-LLMModel=LLaMA2-13B, Sparsity=40%2025.12 | 10.79 | |
| FLAPModel=LLaMA2-7B, Sparsity=40%2025.12 | 11.49 | |
| OBCModel=LLaMA2-13B, Sparsity=40%2025.12 | 11.67 | |
| SliceGPTModel=LLaMA2-7B, Sparsity=40%2025.12 | 12.8 | |
| SVD-LLMModel=LLaMA1-7B, Sparsity=40%2025.12 | 13.85 | |
| SliceGPTModel=LLaMA1-7B, Sparsity=40%2025.12 | 15.94 | |
| SVD-LLMModel=LLaMA2-7B, Sparsity=40%2025.12 | 16.14 |