Language Modeling on Topic Domains (Held-out)
6.84Perplexity (PPL)Llama-8B (KOFF)
Evaluation results
| Method | Links | |
|---|---|---|
| Llama-8B (KOFF)Model Backbone=Llama-8B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 6.84 | |
| Llama-8B (unpruned)Model Backbone=Llama-8B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 7.93 | |
| Llama-3B (unpruned)Model Backbone=Llama-3B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 8.74 | |
| Llama-3B (KOFF)Model Backbone=Llama-3B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 8.85 | |
| Qwen-3B (KOFF)Model Backbone=Qwen-3B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 13.75 | |
| Qwen-3B (unpruned)Model Backbone=Qwen-3B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 14.14 | |
| Llama-8B (pruning only)Model Backbone=Llama-8B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 19.78 | |
| Llama-3B (pruning only)Model Backbone=Llama-3B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 24.01 | |
| Qwen-3B (pruning only)Model Backbone=Qwen-3B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 32.41 |