Zero-shot QA and Reasoning on Downstream Suite (PIQA, HS, ARC, WG, RTE, OQA, BoolQ)
80.79PIQA AccuracyMeta-Llama-3-8B Dense
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Meta-Llama-3-8B DenseBase Model=Meta-Llama-3-8B, Compression Config=-2026.03 | 80.79 | 79.17 | 77.69 | 53.33 | 72.85 | 69.68 | 45 | 81.44 | 69.99 | |
| 3BASiL-TMBase Model=Meta-Llama-3-8B, Compression Config=50%+1282026.03 | 79.65 | 76.07 | 75.84 | 47.78 | 71.98 | 70.4 | 44.2 | 80.7 | 68.33 | |
| 3BASiLBase Model=Meta-Llama-3-8B, Compression Config=50%+1282026.03 | 79.54 | 76.69 | 74.75 | 48.72 | 72.69 | 67.87 | 43 | 80.24 | 67.94 | |
| Hf-SparseGPTBase Model=Meta-Llama-3-8B, Compression Config=50%+1282026.03 | 79.11 | 75.88 | 75 | 49.4 | 73.32 | 63.18 | 43.8 | 78.32 | 67.25 | |
| Hf-ALPSBase Model=Meta-Llama-3-8B, Compression Config=50%+1282026.03 | 78.78 | 76.29 | 76.52 | 51.19 | 73.09 | 60.65 | 40.2 | 81.62 | 67.29 | |
| OATSBase Model=Meta-Llama-3-8B, Compression Config=50%+1282026.03 | 78.4 | 75.32 | 73.99 | 49.15 | 73.8 | 58.84 | 41.8 | 79.42 | 66.34 | |
| Llama-3.2-1B DenseBase Model=Llama-3.2-1B, Compression Config=-2026.03 | 74.59 | 63.66 | 60.48 | 36.26 | 60.69 | 56.68 | 37.2 | 63.98 | 56.69 | |
| 3BASiLBase Model=Llama-3.2-1B, Compression Config=50%+1282026.03 | 73.83 | 60.42 | 58.04 | 34.47 | 60.38 | 53.79 | 36.8 | 58.2 | 54.49 | |
| 3BASiL-TMBase Model=Llama-3.2-1B, Compression Config=50%+1282026.03 | 73.23 | 60.66 | 59.26 | 34.56 | 61.01 | 59.21 | 36.6 | 64.13 | 56.08 | |
| Hf-SparseGPTBase Model=Llama-3.2-1B, Compression Config=50%+1282026.03 | 72.91 | 59.04 | 56.82 | 33.11 | 58.88 | 57.76 | 35 | 57.03 | 53.82 | |
| Hf-ALPSBase Model=Llama-3.2-1B, Compression Config=50%+1282026.03 | 72.8 | 59.92 | 57.62 | 33.11 | 58.64 | 55.96 | 35.2 | 59.66 | 54.11 | |
| OATSBase Model=Llama-3.2-1B, Compression Config=50%+1282026.03 | 71.71 | 57.96 | 57.28 | 33.79 | 59.98 | 52.71 | 33 | 63.94 | 53.8 |