Zero-shot Evaluation on MMLU, HellaSwag, PIQA, BoolQ, WinoGrande, ARC-E, ARC-C, and OBQA (500 samples/task)
74.4MMLUUnpruned
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| UnprunedBackbone=Llama-3.1-70B, Sparsity=0%, Samples per task=5002026.04 | 74.4 | 71.4 | 85.4 | 87 | 67.4 | 91.6 | 67.2 | 89 | 79.2 | |
| SCAR-ConnBackbone=Llama-3.1-70B, Sparsity=50% FFN, Samples per task=5002026.04 | 44 | 49 | 73 | 72.8 | 53.2 | 65.2 | 37.6 | 43.2 | 54.8 | |
| SCAR-LPBackbone=Llama-3.1-70B, Sparsity=50% FFN, Samples per task=5002026.04 | 43.4 | 49.4 | 73 | 73.4 | 53 | 64.8 | 37.4 | 43 | 54.6 | |
| SCAR-ProtBackbone=Llama-3.1-70B, Sparsity=50% FFN, Samples per task=5002026.04 | 42.4 | 48.8 | 74 | 72.2 | 53.2 | 65.4 | 38 | 43 | 54.6 | |
| Wanda (chan.)Backbone=Llama-3.1-70B, Sparsity=50% FFN, Samples per task=5002026.04 | 34.2 | 49.2 | 77.2 | 69 | 58 | 60.4 | 35.4 | 40.2 | 53 | |
| Act. L2Backbone=Llama-3.1-70B, Sparsity=50% FFN, Samples per task=5002026.04 | 32.2 | 42.6 | 67.6 | 52.2 | 49.6 | 44 | 25.2 | 30.8 | 43 | |
| MagnitudeBackbone=Llama-3.1-70B, Sparsity=50% FFN, Samples per task=5002026.04 | 28 | 39.6 | 72.6 | 62.2 | 52.4 | 40.8 | 29.4 | 31.8 | 44.6 |