Zero-shot Evaluation Suite (WikiText2, PTB, BoolQ, PIQA, HellaSwag, WinoGrande, ARC, OBQA)
28.99PTB PerplexityLLaMA-2-7B
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LLaMA-2-7BRatio=0%2024.03 | 28.99 | 13.99 | 42.97 | 76.06 | 70.02 | 65.51 | 63.05 | 36.52 | 40.8 | 56.42 | — | |
| HyWIAPruning Ratio=20%, Evaluation Protocol=Zero-shot2024.03 | 29.61 | — | 72.78 | 77.33 | 68.74 | 66.56 | 64.29 | 38.14 | 42 | 61.35 | 16.63 | |
| Vicuna-7BPruning Ratio=0%, Evaluation Protocol=Zero-shot2024.03 | 58.19 | — | 75.66 | 77.8 | 71.05 | 67.64 | 65.02 | 39.93 | 42.2 | 62.76 | 16.23 | |
| LLM-Pruner VectorPruning Ratio=20%, Evaluation Protocol=Zero-shot2024.03 | 74.66 | — | 63.15 | 74.59 | 61.95 | 60.3 | 60.48 | 36.6 | 39.4 | 56.64 | 19.94 | |
| LLM-Pruner Element²Pruning Ratio=20%, Evaluation Protocol=Zero-shot2024.03 | 76.78 | — | 60.4 | 75.63 | 65.45 | 63.22 | 63.05 | 37.71 | 39 | 57.78 | 18.97 | |
| HyWIARatio=50%2024.03 | 174.19 | 66.37 | 38.26 | 70.73 | 51.61 | 53.91 | 49.03 | 30.72 | 36.6 | 47.26 | — | |
| LLM-Pruner VectorRatio=50%2024.03 | 214.25 | 90.01 | 38.26 | 69.53 | 47.98 | 52.49 | 48.44 | 28.16 | 36 | 45.84 | — | |
| LLM-Pruner Element²Ratio=50%2024.03 | 258.44 | 99.63 | 37.21 | 68.77 | 49.31 | 51.28 | 46.51 | 28.5 | 35.2 | 45.25 | — |