Commonsense Reasoning on Commonsense Tasks
81.35BoolQ AccuracyDense
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| DenseBudget=100%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 81.35 | 80.78 | 72.61 | 79.14 | 77.69 | 53.33 | 45 | 69.99 | |
| DenseBudget=100%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 77.74 | 79.11 | 68.97 | 75.98 | 74.62 | 46.25 | 44.2 | 66.7 | |
| RAPBudget=80%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 68.84 | 76.55 | 66.11 | 65.55 | 62.12 | 39.51 | 39.6 | 59.77 | |
| RAPBudget=80%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 62.81 | 73.99 | 63.38 | 65.77 | 60.35 | 36.69 | 36.6 | 57.08 | |
| ShortGPTBudget=80%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 62.17 | 60.12 | 60.38 | 43.7 | 41.25 | 30.12 | 35 | 47.53 | |
| LLMPrunerBudget=80%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 62.17 | 65.02 | 51.93 | 42.32 | 41.33 | 25.5 | 29.4 | 45.37 | |
| SliceGPTBudget=80%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 61.99 | 65.29 | 58.08 | 43.43 | 52.27 | 32.08 | 28.99 | 48.88 | |
| SliceGPTBudget=80%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 61.87 | 65.94 | 54.05 | 42.51 | 54.37 | 31.06 | 27.8 | 48.23 | |
| SliceGPTBudget=60%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 61.04 | 54.56 | 49.49 | 28.99 | 30.68 | 23.46 | 25.4 | 39.09 | |
| RAPBudget=60%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 57.16 | 58.26 | 53.75 | 37.81 | 37.79 | 26.02 | 30.2 | 43 | |
| ShortGPTBudget=80%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 56.82 | 58.59 | 54.85 | 37.71 | 36.99 | 30.2 | 28 | 43.3 | |
| LLMPrunerBudget=60%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 55.96 | 61.15 | 50.83 | 38.09 | 35.06 | 27.13 | 28.4 | 42.38 | |
| FFN-SkipBudget=80%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 55.77 | 51.47 | 50.74 | 25.9 | 25.75 | 25.25 | 28 | 37.56 | |
| ShortGPTBudget=60%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 55.57 | 50.98 | 50.51 | 27.83 | 26.39 | 27.47 | 26.6 | 37.91 | |
| RAPBudget=60%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 52.2 | 56.69 | 50.36 | 31.91 | 33.54 | 24.23 | 27.4 | 39.48 | |
| FFN-SkipBudget=60%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 51.93 | 51.95 | 50.03 | 26.02 | 24.03 | 26.96 | 28.59 | 37.08 | |
| LLMPrunerBudget=80%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 50.63 | 68.82 | 54.14 | 52.66 | 49.33 | 30.2 | 34.59 | 48.63 | |
| MHA-DropBudget=80%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 46.08 | 54.56 | 50.83 | 29.12 | 28.62 | 27.47 | 25.2 | 37.41 | |
| FFN-SkipBudget=60%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 44.83 | 50.64 | 48.6 | 25.86 | 25.72 | 28.92 | 28.19 | 36.13 | |
| FFN-SkipBudget=80%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 42.8 | 49.35 | 51.22 | 26.69 | 27.18 | 28.49 | 26.6 | 36.05 | |
| ShortGPTBudget=60%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 41.44 | 50.98 | 50.03 | 26.74 | 25.46 | 25.34 | 26.8 | 35.26 | |
| SliceGPTBudget=60%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 40.86 | 53.86 | 48.93 | 27.97 | 32.07 | 23.04 | 25.8 | 36.08 | |
| LLMPrunerBudget=60%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 40.48 | 50.05 | 52.33 | 26.26 | 26.77 | 25.94 | 27 | 35.55 | |
| MHA-DropBudget=60%, Base Model=Llama2-7B, Zero-shot=true2025.05 | 37.83 | 49.89 | 49.88 | 25.72 | 26.05 | 25.43 | 26.8 | 34.51 | |
| MHA-DropBudget=60%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 37.83 | 51.9 | 50.12 | 26.04 | 27.86 | 22.95 | 25.8 | 34.64 | |
| MHA-DropBudget=80%, Base Model=Llama3-8B, Zero-shot=true2025.05 | 37.8 | 53.92 | 50.2 | 26.87 | 30.47 | 23.81 | 27.2 | 35.75 |