Large Language Model Pruning Calibration on OIT-PC Evaluation Suite
99.45Generation Score (SGen)C4
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| C4Pruning Algorithm=SparseGPT, Sparsity=30%2026.06 | 99.45 | 100.9 | — | — | 50.09 | |
| IGSPPruning Algorithm=SparseGPT, Sparsity=30%2026.06 | 98.85 | 100.19 | — | — | 49.76 | |
| Mix-BalancedPruning Algorithm=SparseGPT, Sparsity=30%2026.06 | 98.69 | 99.96 | — | — | 49.66 | |
| Mix-BalancedPruning Algorithm=SparseGPT, Sparsity=40%2026.06 | 97.96 | 98.68 | — | — | 49.16 | |
| C4Pruning Algorithm=Wanda, Sparsity=30%2026.06 | 96.92 | 100.38 | — | — | 49.33 | |
| Mix-BalancedPruning Algorithm=Wanda, Sparsity=30%2026.06 | 96.43 | 99.97 | — | — | 49.1 | |
| C4Pruning Algorithm=SparseGPT, Sparsity=40%2026.06 | 96.1 | 98.65 | — | — | 48.69 | |
| MetaMathPruning Algorithm=SparseGPT, Sparsity=30%2026.06 | 94.88 | 100.43 | — | — | 48.83 | |
| C4Pruning Algorithm=Wanda, Sparsity=40%2026.06 | 92.12 | 98.94 | — | — | 47.77 | |
| Mix-BalancedPruning Algorithm=Wanda, Sparsity=40%2026.06 | 91.77 | 98.76 | — | — | 47.63 | |
| C4 (random)Strategy=C4 (random), Sparsity=40%, Pruning Method=Wanda, Base Model=LLaMA-3.1-8B2026.06 | 91.61 | 96.39 | 76.81 | 64.17 | 82.25 | |
| Self-Gen (single-prompt)Strategy=Self-Gen (single-prompt), Sparsity=40%, Pruning Method=Wanda, Base Model=LLaMA-3.1-8B2026.06 | 91.5 | 96.1 | 78.9 | 66.2 | 83.18 | |
| MetaMathPruning Algorithm=SparseGPT, Sparsity=40%2026.06 | 91.28 | 98.46 | — | — | 47.44 | |
| Self-Gen + SGS filteringStrategy=Self-Gen + SGS filtering, Sparsity=40%, Pruning Method=Wanda, Base Model=LLaMA-3.1-8B2026.06 | 91.2 | 96.2 | 79.1 | 67.5 | 83.5 | |
| C4Pruning Algorithm=SparseGPT, Sparsity=50%2026.06 | 91.01 | 97.32 | — | — | 47.08 | |
| IGSP v2 (multi-prompt)Strategy=IGSP v2 (multi-prompt), Sparsity=40%, Pruning Method=Wanda, Base Model=LLaMA-3.1-8B2026.06 | 90.95 | 96.5 | 80.2 | 69.1 | 84.19 | |
| mix_balanced (real data)Strategy=mix_balanced (real data), Sparsity=40%, Pruning Method=Wanda, Base Model=LLaMA-3.1-8B2026.06 | 89.99 | 97.03 | 74.63 | 69.12 | 82.69 | |
| Mix-BalancedPruning Algorithm=SparseGPT, Sparsity=50%2026.06 | 88.37 | 97.38 | — | — | 46.44 | |
| MetaMathPruning Algorithm=Wanda, Sparsity=30%2026.06 | 85.94 | 99.75 | — | — | 46.42 | |
| C4Pruning Algorithm=Wanda, Sparsity=50%2026.06 | 83.29 | 97.14 | — | — | 45.11 | |
| Mix-BalancedPruning Algorithm=Wanda, Sparsity=50%2026.06 | 82.9 | 96.85 | — | — | 44.94 | |
| C4Pruning Algorithm=SparseGPT, Sparsity=60%2026.06 | 78.72 | 96.27 | — | — | 43.75 | |
| Mix-BalancedPruning Algorithm=SparseGPT, Sparsity=60%2026.06 | 78.51 | 94.77 | — | — | 43.32 | |
| MetaMathPruning Algorithm=SparseGPT, Sparsity=50%2026.06 | 77.88 | 96.86 | — | — | 43.69 | |
| MetaMathPruning Algorithm=Wanda, Sparsity=40%2026.06 | 76.76 | 98.05 | — | — | 43.7 | |
| IGSPPruning Algorithm=SparseGPT, Sparsity=60%2026.06 | 71.46 | 95.67 | — | — | 41.78 | |
| MetaMathPruning Algorithm=Wanda, Sparsity=50%2026.06 | 63.63 | 93.65 | — | — | 39.32 | |
| Mix-BalancedPruning Algorithm=Wanda, Sparsity=60%2026.06 | 61.63 | 90.39 | — | — | 38.01 | |
| C4Pruning Algorithm=Wanda, Sparsity=60%2026.06 | 61.02 | 92.27 | — | — | 38.32 | |
| MetaMathPruning Algorithm=SparseGPT, Sparsity=60%2026.06 | 59.4 | 93.35 | — | — | 38.19 | |
| MetaMathPruning Algorithm=Wanda, Sparsity=60%2026.06 | 42.46 | 87.82 | — | — | 32.57 |