Language Modeling on WikiText (val) (Density/Sampling Evaluation)
8.59Perplexity (Dense)Magnitude
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MagnitudeModel=Pythia-12B, Weight Update=×2025.11 | 8.59 | 127.76 | 200,000 | 700,000 | 200,000 | 300,000 | |
| SparseGPTModel=Pythia-12B, Weight Update=✓2025.11 | 8.59 | 8.59 | 8.65 | 8.86 | 9.39 | 11.02 | |
| WandaModel=Pythia-12B, Weight Update=×2025.11 | 8.59 | 8.59 | 8.6 | 8.85 | 9.31 | 11.27 | |
| AutoPruneModel=Pythia-12B, Weight Update=×2025.11 | 8.59 | 8.59 | 8.6 | 8.8 | 9.27 | 11.13 |