Language Modeling on WikiText-2 raw (test)
5.674PerplexityBase
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| BaseModel=Qwen3-VL-8B-Instruct2026.01 | 5.674 | 0 | 0 | |
| SRAModel=Qwen3-VL-8B-Instruct2026.01 | 5.678 | 0.004 | 0.016 | |
| StandardModel=Qwen3-VL-8B-Instruct2026.01 | 6.332 | 0.658 | 1.337 | |
| SRAModel=Qwen3-VL-4B-Instruct2026.01 | 6.382 | -0.024 | 0.044 | |
| BaseModel=Qwen3-VL-4B-Instruct2026.01 | 6.406 | 0 | 0 | |
| StandardModel=Qwen3-VL-4B-Instruct2026.01 | 6.837 | 0.431 | 2.088 | |
| BaseModel=Qwen3-VL-2B-Instruct2026.01 | 7.266 | 0 | 0 | |
| SRAModel=Qwen3-VL-2B-Instruct2026.01 | 7.294 | 0.028 | 0.018 | |
| SparseGPTModel size=175B, Sparsity=50%2023.01 | 8.21 | — | — | |
| DenseModel size=175B, Sparsity=0%2023.01 | 8.35 | — | — | |
| SparseGPTModel size=175B, Sparsity=4:82023.01 | 8.45 | — | — | |
| SparseGPTModel size=175B, Sparsity=2:42023.01 | 8.74 | — | — | |
| StandardModel=Qwen3-VL-2B-Instruct2026.01 | 8.834 | 1.568 | 0.622 | |
| SparseGPTModel size=66B, Sparsity=50%2023.01 | 9.32 | — | — | |
| DenseModel size=66B, Sparsity=0%2023.01 | 9.34 | — | — | |
| DenseModel size=30B, Sparsity=0%2023.01 | 9.56 | — | — | |
| SparseGPTModel size=66B, Sparsity=4:82023.01 | 9.65 | — | — | |
| SparseGPTModel size=30B, Sparsity=50%2023.01 | 9.79 | — | — | |
| SparseGPTModel size=66B, Sparsity=2:42023.01 | 10.09 | — | — | |
| DenseModel size=13B, Sparsity=0%2023.01 | 10.13 | — | — | |
| SparseGPTModel size=30B, Sparsity=4:82023.01 | 10.3 | — | — | |
| DenseModel size=6.7B, Sparsity=0%2023.01 | 10.86 | — | — | |
| SparseGPTModel size=30B, Sparsity=2:42023.01 | 10.9 | — | — | |
| SUBFITSparsity=20%, Backbone=Llama-3.1-8B2026.06 | 10.9 | — | — | |
| SparseGPTModel size=13B, Sparsity=50%2023.01 | 11.17 | — | — | |
| SparseGPTModel size=6.7B, Sparsity=50%2023.01 | 11.55 | — | — | |
| SparseGPTModel size=13B, Sparsity=4:82023.01 | 11.77 | — | — | |
| DenseModel size=2.7B, Sparsity=0%2023.01 | 12.47 | — | — | |
| SparseGPTModel size=6.7B, Sparsity=4:82023.01 | 12.56 | — | — | |
| SparseGPTModel size=13B, Sparsity=2:42023.01 | 12.96 | — | — | |
| SparseGPTModel size=2.7B, Sparsity=50%2023.01 | 13.48 | — | — | |
| SUBFITSparsity=20%, Backbone=Llama-3.2-3B2026.06 | 13.84 | — | — | |
| BaseModel=Ministral-14B-Instruct-25122026.01 | 14.18 | 0 | 0 | |
| SparseGPTModel size=6.7B, Sparsity=2:42023.01 | 14.2 | — | — | |
| SRAModel=Ministral-14B-Instruct-25122026.01 | 14.22 | 0.04 | 0.026 | |
| DenseModel size=1.3B, Sparsity=0%2023.01 | 14.62 | — | — | |
| SparseGPTModel size=2.7B, Sparsity=4:82023.01 | 14.98 | — | — | |
| StandardModel=Ministral-14B-Instruct-25122026.01 | 15.18 | 1 | 0.723 | |
| SparseGPTModel size=2.7B, Sparsity=2:42023.01 | 17.18 | — | — | |
| SparseGPTModel size=1.3B, Sparsity=50%2023.01 | 17.46 | — | — | |
| BaseModel=Ministral-3B-Instruct-25122026.01 | 20.73 | 0 | 0 | |
| StandardModel=Ministral-3B-Instruct-25122026.01 | 20.78 | 0.05 | 0.097 | |
| SRAModel=Ministral-3B-Instruct-25122026.01 | 20.79 | 0.06 | 0.018 | |
| DenseModel size=350M, Sparsity=0%2023.01 | 22 | — | — | |
| SUBFITSparsity=30%, Backbone=Llama-3.1-8B2026.06 | 23.4 | — | — | |
| SUBFITSparsity=30%, Backbone=Llama-3.2-3B2026.06 | 24.35 | — | — | |
| DenseModel size=125M, Sparsity=0%2023.01 | 27.66 | — | — | |
| SparseGPTModel size=350M, Sparsity=50%2023.01 | 31.58 | — | — | |
| AdaPruneModel size=1.3B, Sparsity=50%2023.01 | 32.52 | — | — | |
| SparseGPTModel size=125M, Sparsity=50%2023.01 | 36.85 | — | — | |
| AdaPruneModel size=350M, Sparsity=50%2023.01 | 48.46 | — | — | |
| SUBFITSparsity=37.5%, Backbone=Llama-3.2-3B2026.06 | 58.62 | — | — | |
| AdaPruneModel size=125M, Sparsity=50%2023.01 | 58.66 | — | — | |
| SUBFITSparsity=37.5%, Backbone=Llama-3.1-8B2026.06 | 62.07 | — | — | |
| MagnitudeModel size=350M, Sparsity=50%2023.01 | 97.8 | — | — | |
| MagnitudeModel size=30B, Sparsity=50%2023.01 | 168 | — | — | |
| MagnitudeModel size=125M, Sparsity=50%2023.01 | 193 | — | — | |
| MagnitudeModel size=2.7B, Sparsity=50%2023.01 | 265 | — | — | |
| MagnitudeModel size=6.7B, Sparsity=50%2023.01 | 969 | — | — | |
| MagnitudeModel size=66B, Sparsity=50%2023.01 | 4,200 | — | — | |
| MagnitudeModel size=13B, Sparsity=50%2023.01 | 12,000 | — | — | |
| MagnitudeModel size=1.3B, Sparsity=50%2023.01 | 17,000 | — | — | |
| MagnitudeModel size=175B, Sparsity=50%2023.01 | 43,000 | — | — |