Zero-shot Commonsense Reasoning on CSQA
83.19PIQASLEB-pruned LLaMA2-7B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SLEB-pruned LLaMA2-7BSparsity=10%, Model (GB)=11.0, Bits=162026.02 | 83.19 | 86.82 | 72.01 | 79.97 | 61.86 | 68.9 | 77 | 75.68 | |
| SLEB-pruned LLaMA2-7BSparsity=10%, Model (GB)=2.9, Bits=42026.02 | 80.2 | 86.88 | 71.03 | 76.68 | 58.53 | 62.43 | 74.6 | 72.91 | |
| SLEB-pruned LLaMA2-7BSparsity=20%, Model (GB)=9.9, Bits=162026.02 | 76.77 | 75.07 | 67.2 | 73.53 | 52.73 | 60.62 | 65 | 67.27 | |
| SLEB-pruned LLaMA2-7BSparsity=20%, Model (GB)=2.6, Bits=42026.02 | 72.85 | 72.32 | 65.66 | 68.94 | 47.87 | 55.56 | 60.2 | 63.34 | |
| SLEB-pruned LLaMA2-7BSparsity=10%, Model (GB)=2.2, Bits=32026.02 | 71.16 | 77.15 | 64.48 | 64.73 | 45.73 | 56.75 | 55.2 | 62.17 | |
| SLEB-pruned LLaMA2-7BSparsity=20%, Model (GB)=1.9, Bits=32026.02 | 53.97 | 48.55 | 49.08 | 44.02 | 30.38 | 54.38 | 25.2 | 43.65 |