Natural Language Understanding on (Winogrande, PIQA, WSC, WNLI, SST-2, RTE, QNLI, CB, ARC-e, ARC-c) (test)
75.61Winogrande AccuracyDense
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DenseModel=Llama2-13B, Ratio=0.00%2026.01 | 75.61 | 79.71 | 53.85 | 66.2 | 87.61 | 69.31 | 58.56 | 80.36 | 81.82 | 53.16 | 70.62 | |
| DenseModel=Qwen-14B, Ratio=0.00%2026.01 | 75.3 | 80.01 | 69.23 | 73.24 | 95.07 | 80.51 | 74.81 | 87.5 | 81.14 | 52.47 | 76.93 | |
| DenseModel=Llama2-7B, Ratio=0.00%2026.01 | 72.85 | 78.24 | 54.81 | 59.15 | 85.89 | 66.06 | 54.6 | 60.71 | 79 | 47.78 | 65.91 | |
| DenseModel=Qwen-7B, Ratio=0.00%2026.01 | 71.98 | 79.22 | 73.08 | 64.79 | 94.15 | 83.75 | 74.24 | 76.79 | 79.04 | 48.04 | 74.51 | |
| LLM-PrunerModel=Llama2-13B, Ratio=25.35%2026.01 | 71.74 | 73.07 | 36.54 | 43.66 | 66.86 | 52.71 | 49.9 | 42.86 | 66.08 | 32.42 | 53.58 | |
| SliceGPTModel=Llama2-13B, Ratio=25.28%2026.01 | 70.72 | 63.17 | 44.23 | 43.66 | 51.15 | 52.71 | 50.58 | 41.07 | 65.03 | 33.79 | 51.61 | |
| MI-PRUNModel=Llama2-7B, Ratio=15.32%2026.01 | 69.69 | 73.5 | 63.46 | 60.56 | 83.94 | 60.29 | 50.91 | 62.5 | 67.21 | 39.42 | 63.15 | |
| MI-PRUNModel=Qwen-14B, Ratio=15.58%2026.01 | 69.46 | 72.25 | 81.73 | 71.83 | 95.41 | 80.87 | 69.89 | 83.93 | 67.38 | 43.69 | 73.64 | |
| MI-PRUNModel=Qwen-7B, Ratio=13.11%2026.01 | 69.3 | 73.78 | 63.46 | 61.97 | 94.04 | 71.84 | 50.54 | 62.5 | 66.25 | 39.93 | 65.36 | |
| ShortGPTModel=Llama2-7B, Ratio=15.32%2026.01 | 67.72 | 71.6 | 36.54 | 43.66 | 49.2 | 53.43 | 50.19 | 39.29 | 65.66 | 39.42 | 51.67 | |
| ShortGPTModel=Qwen-7B, Ratio=13.11%2026.01 | 67.4 | 73.01 | 63.46 | 63.38 | 93.12 | 80.87 | 50.54 | 60.71 | 62.96 | 35.84 | 65.13 | |
| ShortGPTModel=Qwen-14B, Ratio=15.58%2026.01 | 67.4 | 70.18 | 36.54 | 71.83 | 94.15 | 80.51 | 49.46 | 87.5 | 63.72 | 39.33 | 66.06 | |
| LLM-PrunerModel=Llama2-7B, Ratio=15.30%2026.01 | 64.72 | 76.06 | 36.54 | 56.34 | 65.83 | 52.71 | 51.2 | 46.43 | 64.86 | 36.69 | 55.14 | |
| ShortGPTModel=Llama2-13B, Ratio=24.37%2026.01 | 64.17 | 71.93 | 58.65 | 59.15 | 50.57 | 68.95 | 49.83 | 57.14 | 50.17 | 40.53 | 57.11 | |
| SliceGPTModel=Llama2-7B, Ratio=15.34%2026.01 | 63.06 | 67.03 | 63.46 | 45.07 | 52.64 | 59.57 | 50.63 | 42.86 | 55.26 | 34.56 | 53.41 | |
| MI-PRUNModel=Llama2-13B, Ratio=24.37%2026.01 | 59.59 | 72.36 | 59.62 | 66.2 | 70.87 | 69.31 | 50.61 | 78.57 | 60.02 | 40.27 | 62.74 |