Zero-shot Common-sense Reasoning Suite (BoolQ, PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, OBQA)
70.185Avg AccuracyLLaMA3-8B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LLaMA3-8BPruned Rates=0%, Model=LLaMA3-8B, Evaluation Protocol=Zero-shot2025.02 | 70.185 | 81.101 | 79.489 | 79.167 | 73.402 | 80.093 | 53.242 | 44.8 | |
| SWMPruned Rates=20%, Model=LLaMA3-8B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 66.954 | 76.789 | 77.639 | 73.77 | 71.744 | 76.599 | 50.939 | 41.2 | |
| LLaMA2-7BPruned Rates=0%, Model=LLaMA2-7B, Evaluation Protocol=Zero-shot2025.02 | 66.825 | 77.706 | 78.074 | 76.021 | 69.14 | 76.305 | 46.331 | 44.2 | |
| LLM-PrunerPruned Rates=20%, Model=LLaMA3-8B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 65.941 | 74.037 | 79.489 | 74.268 | 70.324 | 74.285 | 46.587 | 42.6 | |
| Vicuna-7B-v1.3Pruned Rates=0%, Model=Vicuna-7B-v1.3, Evaluation Protocol=Zero-shot2025.02 | 65.902 | 78.104 | 77.312 | 73.939 | 69.376 | 74.327 | 44.454 | 43.8 | |
| Shortened-LLMPruned Rates=20%, Model=LLaMA3-8B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 63.316 | 66.208 | 78.074 | 72.695 | 62.747 | 75.295 | 44.795 | 43.4 | |
| LLaMA2-7BRatio=100%2026.03 | 62.91 | 71.1 | 78.4 | 72.96 | 67.25 | 69.36 | 40.53 | 40.8 | |
| SWMPruned Rates=20%, Model=Vicuna-7B-v1.3, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 62.628 | 77.431 | 74.755 | 69.04 | 68.745 | 69.318 | 38.908 | 40.2 | |
| SWMPruned Rates=20%, Model=LLaMA2-7B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 61.944 | 69.45 | 73.667 | 70.484 | 67.088 | 69.108 | 41.212 | 42.6 | |
| FLAPPruned Rates=20%, Model=Vicuna-7B-v1.3, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 61.664 | 73.52 | 74.81 | 68.76 | 66.46 | 69.11 | 38.99 | 40 | |
| Shortened-LLMPruned Rates=20%, Model=Vicuna-7B-v1.3, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 60.865 | 72.355 | 74.701 | 67.576 | 64.562 | 70.034 | 38.225 | 38.6 | |
| Wanda-spPruned Rates=20%, Model=LLaMA2-7B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 60.747 | 62.6 | 76.44 | 70.66 | 63.77 | 69.61 | 42.15 | 40 | |
| LLM-PrunerPruned Rates=20%, Model=Vicuna-7B-v1.3, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 59.543 | 67.645 | 76.115 | 66.66 | 63.931 | 65.446 | 36.604 | 40.4 | |
| LLM-PrunerPruned Rates=20%, Model=LLaMA2-7B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 59.268 | 63.731 | 77.476 | 67.128 | 61.878 | 65.783 | 38.481 | 40.4 | |
| Wanda-spPruned Rates=20%, Model=Vicuna-7B-v1.3, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 58.976 | 63.27 | 73.78 | 68.62 | 63.93 | 67.21 | 38.82 | 37.2 | |
| Shortened-LLMPruned Rates=20%, Model=LLaMA2-7B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 58.445 | 61.56 | 76.061 | 67.994 | 58.8 | 68.813 | 37.884 | 38 | |
| FLAPPruned Rates=20%, Model=LLaMA2-7B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 58.059 | 72.05 | 73.39 | 64.69 | 64.72 | 62.25 | 32.51 | 36.8 | |
| SWMPruned Rates=35%, Model=LLaMA3-8B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 57.652 | 67.554 | 73.83 | 61.472 | 62.747 | 64.352 | 36.007 | 37.6 | |
| SLEBPruned Rates=20%, Model=LLaMA3-8B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 57.506 | 62.171 | 73.286 | 64.748 | 63.062 | 64.562 | 37.713 | 37 | |
| Shortened-LLMPruned Rates=35%, Model=LLaMA3-8B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 56.871 | 63.18 | 72.851 | 62.985 | 58.09 | 66.877 | 37.116 | 37 | |
| SLEBPruned Rates=20%, Model=LLaMA2-7B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 56.733 | 62.875 | 73.939 | 63.951 | 59.747 | 63.468 | 35.154 | 38 | |
| LLM-PrunerPruned Rates=35%, Model=LLaMA3-8B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 56.557 | 64.465 | 74.048 | 61.8 | 59.353 | 64.646 | 34.386 | 37.2 | |
| SWMPruned Rates=35%, Model=Vicuna-7B-v1.3, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 56.25 | 69.235 | 70.294 | 60.705 | 62.273 | 60.227 | 33.618 | 37.4 | |
| LLM-PrunerPruned Rates=35%, Model=Vicuna-7B-v1.3, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 54.596 | 63.976 | 73.069 | 59.56 | 58.564 | 56.524 | 32.679 | 37.8 | |
| FLAPPruned Rates=35%, Model=Vicuna-7B-v1.3, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 54.203 | 57.86 | 69.64 | 59.12 | 63.3 | 57.83 | 35.67 | 36 | |
| Shortened-LLMPruned Rates=35%, Model=Vicuna-7B-v1.3, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 53.464 | 64.281 | 70.783 | 56.722 | 57.38 | 59.596 | 31.485 | 34 | |
| LLM-PrunerPruned Rates=35%, Model=LLaMA2-7B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 52.366 | 45.26 | 74.76 | 60.29 | 59.35 | 57.28 | 32.42 | 37.2 | |
| Shortened-LLMPruned Rates=35%, Model=LLaMA2-7B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 52.317 | 62.171 | 71.926 | 54.8 | 51.776 | 59.259 | 30.887 | 35.4 | |
| SWMPruned Rates=35%, Model=LLaMA2-7B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 51.869 | 63.119 | 65.452 | 56.503 | 58.879 | 52.02 | 31.911 | 35.2 | |
| FLAPPruned Rates=35%, Model=LLaMA2-7B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 51.194 | 66.97 | 67.85 | 52.1 | 61.48 | 49.49 | 28.07 | 32.4 | |
| Wanda-spPruned Rates=35%, Model=LLaMA2-7B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 50.35 | 59.79 | 68.82 | 53.14 | 54.06 | 52.27 | 31.57 | 32.8 | |
| GradMAPBackbone=LLaMA3.1-8B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 22026.02 | 48.92 | — | — | — | — | — | — | — | |
| GradMAPBackbone=LLaMA3.1-8B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 12026.02 | 48.68 | — | — | — | — | — | — | — | |
| SLEBPruned Rates=35%, Model=LLaMA2-7B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 47.499 | 54.801 | 67.41 | 46.545 | 53.197 | 48.527 | 29.01 | 33 | |
| SLEBPruned Rates=35%, Model=LLaMA3-8B, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 46.147 | 59.755 | 64.635 | 45.061 | 51.539 | 47.306 | 27.133 | 27.6 | |
| ShortGPTBackbone=LLaMA3.1-8B, Compression Ratio=25%, Zero-shot=true2026.02 | 45.5 | — | — | — | — | — | — | — | |
| Wanda-spPruned Rates=35%, Model=Vicuna-7B-v1.3, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 43.991 | 50.76 | 60.45 | 43.06 | 55.96 | 43.52 | 26.19 | 28 | |
| GPrune-LLM (Wanda-sp)Ratio=40%, Base Method=Wanda-sp2026.03 | 41.86 | 61.25 | 56.53 | 31.47 | 51.3 | 31.82 | 29.27 | 31.4 | |
| GPrune-LLM (FLAP)Ratio=40%, Base Method=FLAP2026.03 | 41.26 | 45.41 | 59.52 | 36.42 | 49.72 | 36.95 | 27.39 | 33.4 | |
| GradMAPBackbone=Baichuan2-7B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 22026.02 | 39.6 | — | — | — | — | — | — | — | |
| GradMAPBackbone=Baichuan2-7B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 12026.02 | 39.21 | — | — | — | — | — | — | — | |
| FLAPRatio=40%2026.03 | 38.96 | 43.09 | 58.22 | 30.23 | 48.22 | 33.54 | 26.45 | 33 | |
| LLM-PrunerRatio=40%2026.03 | 38.85 | 59.51 | 55.88 | 26.67 | 47.91 | 27.99 | 25 | 29 | |
| ShortGPTBackbone=Baichuan2-7B, Compression Ratio=25%, Zero-shot=true2026.02 | 38.31 | — | — | — | — | — | — | — | |
| GradMAPBackbone=OPT-6.7B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 22026.02 | 38.1 | — | — | — | — | — | — | — | |
| GradMAPBackbone=OPT-6.7B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 12026.02 | 37.66 | — | — | — | — | — | — | — | |
| SLEBBackbone=LLaMA3.1-8B, Compression Ratio=25%, Zero-shot=true2026.02 | 37.62 | — | — | — | — | — | — | — | |
| GradMAPBackbone=Qwen2.5-7B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 22026.02 | 37.51 | — | — | — | — | — | — | — | |
| GradMAPBackbone=Qwen2.5-7B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 12026.02 | 37.32 | — | — | — | — | — | — | — | |
| Wanda-spRatio=40%2026.03 | 37.1 | 47.55 | 52.77 | 25.98 | 49.09 | 26.85 | 25.85 | 31.6 | |
| SLEBBackbone=OPT-6.7B, Compression Ratio=25%, Zero-shot=true2026.02 | 36.58 | — | — | — | — | — | — | — | |
| SLEBPruned Rates=20%, Model=Vicuna-7B-v1.3, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 36.449 | 46.758 | 51.85 | 26.17 | 51.223 | 25.505 | 28.84 | 24.8 | |
| SLEBBackbone=Qwen2.5-7B, Compression Ratio=25%, Zero-shot=true2026.02 | 35.41 | — | — | — | — | — | — | — | |
| SLEBPruned Rates=35%, Model=Vicuna-7B-v1.3, Pruning Type=depth, Evaluation Protocol=Zero-shot2025.02 | 35.399 | 37.829 | 53.264 | 25.921 | 49.961 | 25.926 | 29.096 | 25.8 | |
| FLAPPruned Rates=20%, Model=LLaMA3-8B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 34.853 | 37.83 | 52.18 | 26.36 | 49.96 | 26.81 | 24.83 | 26 | |
| ShortGPTBackbone=Qwen2.5-7B, Compression Ratio=25%, Zero-shot=true2026.02 | 34.49 | — | — | — | — | — | — | — | |
| FLAPPruned Rates=35%, Model=LLaMA3-8B, Pruning Type=width, Evaluation Protocol=Zero-shot2025.02 | 34.147 | 37.83 | 52.34 | 26.05 | 47.99 | 24.79 | 24.83 | 25.2 | |
| GradMAPBackbone=LLaMA3.2-3B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 22026.02 | 32.75 | — | — | — | — | — | — | — | |
| GradMAPBackbone=LLaMA3.2-3B, Compression Ratio=25%, Zero-shot=true, Stage=Stage 12026.02 | 32.69 | — | — | — | — | — | — | — | |
| SLEBBackbone=Baichuan2-7B, Compression Ratio=25%, Zero-shot=true2026.02 | 32.2 | — | — | — | — | — | — | — | |
| ShortGPTBackbone=LLaMA3.2-3B, Compression Ratio=25%, Zero-shot=true2026.02 | 30.46 | — | — | — | — | — | — | — | |
| SLEBBackbone=LLaMA3.2-3B, Compression Ratio=25%, Zero-shot=true2026.02 | 30.1 | — | — | — | — | — | — | — | |
| ShortGPTBackbone=OPT-6.7B, Compression Ratio=25%, Zero-shot=true2026.02 | 29.26 | — | — | — | — | — | — | — |