Zero-shot Accuracy on QA/Reasoning Suite (BoolQ, WinoG., PIQA, OBQA, HellaS., ARC-e, ARC-c) (test)
83.76BoolQ AccuracySparseGPT
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SparseGPTSparsity=50%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 83.76 | 72.3 | 79.54 | 40.8 | 75.61 | 74.37 | 44.97 | 67.34 | |
| ROSESparsity=40%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 82.81 | 72.77 | 81.61 | 42.6 | 78.63 | 75.88 | 49.49 | 69.11 | |
| SparseGPTSparsity=40%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 82.75 | 72.45 | 81.34 | 42.6 | 78.49 | 76.01 | 48.55 | 68.88 | |
| ROSESparsity=30%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 82.54 | 72.45 | 81.66 | 43.2 | 79.95 | 77.53 | 51.54 | 69.84 | |
| SparseGPTSparsity=30%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 82.45 | 72.69 | 81.66 | 43 | 79.95 | 77.86 | 51.79 | 69.91 | |
| ROSESparsity=50%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 82.39 | 71.67 | 79.43 | 40.6 | 75.61 | 74.49 | 46.42 | 67.23 | |
| DenseSparsity=0%, Backbone=Mistral-7B, Pruning Type=None, Evaluation Protocol=Zero-shot2026.03 | 82.14 | 73.8 | 82.26 | 44.2 | 80.42 | 78.2 | 52.3 | 70.47 | |
| SparseGPTSparsity=60%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 77.83 | 68.11 | 76.82 | 38.8 | 67.25 | 66.58 | 39.76 | 62.16 | |
| ROSESparsity=60%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 76.27 | 67.32 | 76.39 | 38.2 | 67.47 | 66.37 | 37.63 | 61.38 | |
| SparseGPTSparsity=70%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 67.25 | 58.25 | 65.56 | 31.2 | 46.64 | 46.8 | 27.22 | 48.99 | |
| ROSESparsity=70%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 65.14 | 60.14 | 66.1 | 30 | 46.94 | 48.78 | 27.56 | 49.24 | |
| ROSESparsity=80%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 59.76 | 50.2 | 53.86 | 26.6 | 29.6 | 30.05 | 20.82 | 38.7 | |
| SparseGPTSparsity=80%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 53.91 | 50.75 | 54.13 | 26.8 | 29.56 | 29.63 | 20.9 | 37.95 | |
| ROSESparsity=90%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 37.86 | 47.67 | 51.8 | 26.8 | 27.06 | 28.37 | 24.74 | 34.9 | |
| SparseGPTSparsity=90%, Backbone=Mistral-7B, Pruning Type=Unstructured, Evaluation Protocol=Zero-shot2026.03 | 37.83 | 48.93 | 52.29 | 25.8 | 27.06 | 28.37 | 25.09 | 35.05 |