Natural Language Understanding on Aggregate NLP Suite 7 Benchmarks
72.5Average AccuracyNo pruning
Evaluation Results
| Method | Links | |
|---|---|---|
| No pruningSparsity=0, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 72.5 | |
| Magnitude-DimSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 65.9 | |
| DIETSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 64.6 | |
| DIETSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 51.6 | |
| Magnitude-DimSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 51 | |
| PuDDingSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 49.7 | |
| PuDDingSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 48.9 | |
| SliceGPTSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 35.3 | |
| SliceGPTSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 35.2 |