Zero-shot Language Understanding on Reasoning Suite (BoolQ, WinoG., PIQA, OBQA, HellaS., ARC-e, ARC-c)
82.63BoolQ AccuracySparseGPT
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SparseGPTBackbone=Mistral-7B, Sparsity=25%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 82.63 | 72.77 | 81.99 | 43.6 | 80.12 | 77.78 | 51.96 | 70.12 | |
| ROSEBackbone=Mistral-7B, Sparsity=25%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 82.51 | 73.56 | 81.94 | 43.8 | 80.06 | 77.95 | 51.79 | 70.23 | |
| ROSEBackbone=Mistral-7B, Sparsity=50%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 82.45 | 70.72 | 79.33 | 40.6 | 75.25 | 73.91 | 45.82 | 66.87 | |
| SparseGPTBackbone=Mistral-7B, Sparsity=50%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 82.17 | 70.64 | 79.33 | 40.8 | 75.29 | 73.99 | 44.54 | 66.68 | |
| SparseGPTBackbone=Mistral-7B, Sparsity=0%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 81.99 | 73.88 | 81.77 | 45 | 80.4 | 78.49 | 52.3 | 70.55 | |
| ROSEModel Backbone=Mistral-7B, Sparsity=50%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 81.96 | 69.93 | 78.18 | 39 | 72.88 | 71.93 | 43.94 | 65.4 | |
| ROSEBackbone=Mistral-7B, Sparsity=0%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 81.96 | 73.48 | 81.94 | 44.8 | 80.36 | 78.45 | 52.3 | 70.47 | |
| SparseGPTModel Backbone=Mistral-7B, Sparsity=25%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 81.44 | 69.93 | 79.71 | 43.2 | 77.35 | 73.61 | 47.35 | 67.51 | |
| ROSEModel Backbone=Mistral-7B, Sparsity=0%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 80.89 | 72.77 | 81.34 | 43.2 | 78.78 | 76.01 | 50.17 | 69.02 | |
| SparseGPTModel Backbone=Mistral-7B, Sparsity=50%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 80.86 | 69.38 | 78.13 | 39.6 | 71.23 | 71.3 | 43.26 | 64.82 | |
| ROSEModel Backbone=Mistral-7B, Sparsity=25%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 80.64 | 71.11 | 80.96 | 44 | 78.51 | 75.8 | 48.98 | 68.57 | |
| SparseGPTModel Backbone=Mistral-7B, Sparsity=0%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 80.61 | 71.43 | 80.3 | 43.2 | 77.43 | 74.66 | 50 | 68.23 | |
| ROSEBackbone=LLaMA2-7B, Sparsity=0%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 77.4 | 68.75 | 79.05 | 44.2 | 76.01 | 74.54 | 46.16 | 66.59 | |
| SparseGPTBackbone=LLaMA2-7B, Sparsity=0%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 77.34 | 68.82 | 79 | 44.2 | 76 | 74.33 | 46.25 | 66.56 | |
| ROSEBackbone=LLaMA2-7B, Sparsity=25%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 77.03 | 70.24 | 79.16 | 45 | 76.25 | 72.43 | 46.33 | 66.63 | |
| SparseGPTBackbone=LLaMA2-7B, Sparsity=25%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 76.97 | 69.61 | 78.84 | 44.6 | 76.26 | 73.06 | 46.5 | 66.55 | |
| SparseGPTModel Backbone=LLaMA2-7B, Sparsity=0%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 76.42 | 68.19 | 78.51 | 42.2 | 74.6 | 71.97 | 44.28 | 65.14 | |
| ROSEModel Backbone=LLaMA2-7B, Sparsity=50%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 75.99 | 68.35 | 77.48 | 40.6 | 69.78 | 65.45 | 41.04 | 62.67 | |
| ROSEModel Backbone=LLaMA2-7B, Sparsity=0%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 75.84 | 67.88 | 78.45 | 42.4 | 74.6 | 72.22 | 44.62 | 65.14 | |
| SparseGPTBackbone=LLaMA2-7B, Sparsity=50%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 75.78 | 70.17 | 77.15 | 41.8 | 71.11 | 67.85 | 41.47 | 63.62 | |
| ROSEBackbone=LLaMA2-7B, Sparsity=50%, Quantization=8-bit, Evaluation=zero-shot2026.03 | 75.54 | 69.3 | 76.88 | 42.8 | 70.85 | 66.54 | 41.55 | 63.35 | |
| SparseGPTModel Backbone=LLaMA2-7B, Sparsity=25%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 74.92 | 68.9 | 77.42 | 40.6 | 75.15 | 69.36 | 42.66 | 64.14 | |
| ROSEModel Backbone=LLaMA2-7B, Sparsity=25%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 73.73 | 68.51 | 78.18 | 40.6 | 75.38 | 70.83 | 46.5 | 64.81 | |
| SparseGPTModel Backbone=LLaMA2-7B, Sparsity=50%, Quantization=4-bit, Evaluation Protocol=Zero-shot2026.03 | 73.61 | 69.06 | 75.9 | 40.6 | 69.31 | 63.51 | 39.93 | 61.7 |