Aggregate NLP Evaluation on NLP Benchmark Suite Average
64Average AccuracyNo pruning (Original)
Evaluation results
| Method | Links | |
|---|---|---|
| No pruning (Original)Backbone=Gemma-2 2B, Sparsity=0, Evaluation Protocol=Zero-shot2026.03 | 64 | |
| DIETBackbone=Gemma-2 2B, Sparsity=10%, Evaluation Protocol=Zero-shot2026.03 | 56.3 | |
| Magnitude-DimBackbone=Gemma-2 2B, Sparsity=10%, Evaluation Protocol=Zero-shot2026.03 | 46.6 | |
| DIETBackbone=Gemma-2 2B, Sparsity=20%, Evaluation Protocol=Zero-shot2026.03 | 45 | |
| Magnitude-DimBackbone=Gemma-2 2B, Sparsity=20%, Evaluation Protocol=Zero-shot2026.03 | 37.2 | |
| PuDDingBackbone=Gemma-2 2B, Sparsity=10%, Evaluation Protocol=Zero-shot2026.03 | 35.9 | |
| PuDDingBackbone=Gemma-2 2B, Sparsity=20%, Evaluation Protocol=Zero-shot2026.03 | 35.8 | |
| SliceGPTBackbone=Gemma-2 2B, Sparsity=20%, Evaluation Protocol=Zero-shot2026.03 | 35 | |
| SliceGPTBackbone=Gemma-2 2B, Sparsity=10%, Evaluation Protocol=Zero-shot2026.03 | 35 |