Language Modeling on C4 v1 (val)
11.07PerplexityDense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseSparsity=0%, Backbone=Llama-3.2-3B2026.06 | 11.07 | |
| SUBFITSparsity=20%, Backbone=Llama-3.1-8B2026.06 | 15.6 | |
| SUBFITSparsity=20%, Backbone=Llama-3.2-3B2026.06 | 19.06 | |
| SUBFITSparsity=30%, Backbone=Llama-3.2-3B2026.06 | 26.23 | |
| SUBFITSparsity=30%, Backbone=Llama-3.1-8B2026.06 | 27.27 | |
| SUBFITSparsity=20%, Backbone=Qwen3-4B2026.06 | 32.63 | |
| SUBFITSparsity=37.5%, Backbone=Llama-3.1-8B2026.06 | 43.69 | |
| SUBFITSparsity=37.5%, Backbone=Llama-3.2-3B2026.06 | 45.02 | |
| SUBFITSparsity=30%, Backbone=Qwen3-4B2026.06 | 58.99 | |
| SUBFITSparsity=37.5%, Backbone=Qwen3-4B2026.06 | 99.91 |