Zero-shot Evaluation on Mean tasks zero-shot (test)
68.36Mean Zero-Shot AccuracyDense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseModel=Qwen3-14B, Sparsity=None2026.05 | 68.36 | |
| DenseModel=Qwen3-8B, Sparsity=None2026.05 | 65.73 | |
| SparseForgeModel=Qwen3-14B, Sparsity=2:4 semi-structured2026.05 | 65.44 | |
| SparseForgeModel=Qwen3-8B, Sparsity=2:4 semi-structured2026.05 | 63.31 | |
| DenseModel=DeepSeek-MoE, Sparsity=None2026.05 | 59.54 | |
| SparseForgeModel=DeepSeek-MoE, Sparsity=2:4 semi-structured2026.05 | 58.57 | |
| DenseModel=Qwen3-1.7B, Sparsity=None2026.05 | 56.51 | |
| SparseForgeModel=Qwen3-1.7B, Sparsity=2:4 semi-structured2026.05 | 53.33 | |
| DenseModel=OPT-2.7B, Sparsity=None2026.05 | 47.76 | |
| SparseForgeModel=OPT-2.7B, Sparsity=2:4 semi-structured2026.05 | 46.67 | |
| DenseModel=GPT2-XL, Sparsity=None2026.05 | 45.49 | |
| SparseForgeModel=GPT2-XL, Sparsity=2:4 semi-structured2026.05 | 44.34 | |
| DenseModel=GPT2-L, Sparsity=None2026.05 | 42.76 | |
| SparseForgeModel=GPT2-L, Sparsity=2:4 semi-structured2026.05 | 42.1 | |
| DenseModel=GPT2-M, Sparsity=None2026.05 | 40.97 | |
| SparseForgeModel=GPT2-M, Sparsity=2:4 semi-structured2026.05 | 40.31 |