Commonsense Reasoning on BoolQ, PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, OBQA (test)
73.54BoolQ AccuracyFlexora
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| FlexoraBackbone=Llama-7B2024.08 | 73.54 | 71.93 | 85.28 | 74.11 | 71.22 | 45.64 | 39.86 | 65.94 | |
| LoRABackbone=Llama-7B2024.08 | 67.76 | 69.8 | 76.1 | 67.01 | 67.21 | 35.23 | 38.6 | 60.24 | |
| LoRAShearBackbone=Llama-7B, Pruning Ratio=0.52024.08 | 63.4 | 72.15 | 49.83 | 56.4 | 49.45 | 34.31 | 35.86 | 51.63 | |
| Pre-trainedBackbone=Llama-7B2024.08 | 57.98 | 60.94 | 34.35 | 52.25 | 31.82 | 27.3 | 35.8 | 42.92 |