Question Answering on OBQA (acc_norm)
41.8Accuracy (Normalized)No pruning (Original)
Evaluation Results
| Method | Links | |
|---|---|---|
| No pruning (Original)Backbone=Gemma-2 2B, Sparsity=0, Evaluation Protocol=Zero-shot2026.03 | 41.8 | |
| WandaBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 38.8 | |
| AutoPruneBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 38.8 | |
| DenseBackbone=LLaMA-1, Model Parameters=65B, Sparsity=0%, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 38.2 | |
| SparseGPTBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 37.2 | |
| DenseBackbone=LLaMA-1, Model Parameters=30B, Sparsity=0%, Evaluation Protocol=zero-shot2025.11 | 36 | |
| DIETBackbone=Gemma-2 2B, Sparsity=10%, Evaluation Protocol=Zero-shot2026.03 | 35.4 | |
| AutoPruneBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 35.2 | |
| SparseGPTBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 35 | |
| MagnitudeBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 35 | |
| WandaBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 34.6 | |
| DenseBackbone=LLaMA-1, Model Parameters=7B, Sparsity=0%, Evaluation Protocol=zero-shot2025.11 | 34.4 | |
| DenseBackbone=LLaMA-1, Model Parameters=13B, Sparsity=0%, Evaluation Protocol=zero-shot2025.11 | 33.2 | |
| WandaBackbone=LLaMA-1, Model Parameters=13B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 32.2 | |
| AutoPruneBackbone=LLaMA-1, Model Parameters=13B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 32.2 | |
| Magnitude-DimBackbone=Gemma-2 2B, Sparsity=10%, Evaluation Protocol=Zero-shot2026.03 | 31.2 | |
| SparseGPTBackbone=LLaMA-1, Model Parameters=13B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 31.2 | |
| PuDDingBackbone=Gemma-2 2B, Sparsity=10%, Evaluation Protocol=Zero-shot2026.03 | 30 | |
| SparseGPTBackbone=LLaMA-1, Model Parameters=7B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 30 | |
| AutoPruneBackbone=LLaMA-1, Model Parameters=7B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 30 | |
| MagnitudeBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 29 | |
| WandaBackbone=LLaMA-1, Model Parameters=7B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 28.8 | |
| DIETBackbone=Gemma-2 2B, Sparsity=20%, Evaluation Protocol=Zero-shot2026.03 | 27.6 | |
| MagnitudeBackbone=LLaMA-1, Model Parameters=13B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 27.2 | |
| SliceGPTBackbone=Gemma-2 2B, Sparsity=20%, Evaluation Protocol=Zero-shot2026.03 | 27 | |
| PuDDingBackbone=Gemma-2 2B, Sparsity=20%, Evaluation Protocol=Zero-shot2026.03 | 27 | |
| SliceGPTBackbone=Gemma-2 2B, Sparsity=10%, Evaluation Protocol=Zero-shot2026.03 | 27 | |
| Magnitude-DimBackbone=Gemma-2 2B, Sparsity=20%, Evaluation Protocol=Zero-shot2026.03 | 25 | |
| MagnitudeBackbone=LLaMA-1, Model Parameters=7B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 22.4 |