Question Answering on Race, Winogrande, PIQA, BoolQ, OpenBookQA, and SciQ
70Average AccuracyLlama 3 8B Instruct
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama 3 8B InstructBackbone=Llama 3 8B Instruct, Pruning ratio=0%, Linear transform=None2025.05 | 70 | 100 | |
| ReplaceMeCompression ratio=12.5%2025.05 | 64.6 | — | |
| ReplaceMeBackbone=Llama 3 8B Instruct, Pruning ratio=25%, Linear transform=Linear (LS)2025.05 | 63 | 89.9 | |
| ReplaceMeBackbone=Llama 3 8B Instruct, Pruning ratio=25%, Linear transform=Linear (Cosine)2025.05 | 63 | 90.9 | |
| ReplaceMeBackbone=Llama 3 8B Instruct, Pruning ratio=25%, Linear transform=Multi_LT_NC (Cosine)2025.05 | 63 | 90 | |
| ReplaceMeCompression ratio=25%2025.05 | 60.7 | — | |
| LLMPrunerBackbone=Llama 3 8B Instruct, Pruning ratio=25%, Linear transform=None2025.05 | 60 | 85.3 | |
| UIDLCompression ratio=12.5%2025.05 | 59.9 | — | |
| SVD-LLMBackbone=Llama 3 8B Instruct, Pruning ratio=25%, Linear transform=None2025.05 | 59 | 85.3 | |
| UIDLBackbone=Llama 3 8B Instruct, Pruning ratio=25%, Linear transform=Identity2025.05 | 58 | 82.5 | |
| ReplaceMeCompression ratio=37.5%2025.05 | 51 | — | |
| UIDLCompression ratio=25%2025.05 | 50.4 | — | |
| UIDLCompression ratio=37.5%2025.05 | 41 | — | |
| ReplaceMeCompression ratio=50%2025.05 | 38.6 | — | |
| UIDLCompression ratio=50%2025.05 | 30.4 | — |