Multiple Choice Classification on OpenBookQA, ARC-Easy, WinoGrande, HellaSwag, PIQA, and MathQA
35Accuracy (OpenBookQA)Original
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OriginalBackbone Model=Llama-3.1-8B, Compression Rate=None2024.11 | 35 | 80 | 73 | 60 | 80 | 40 | 61 | |
| OriginalBackbone Model=Llama-7B, Compression Rate=None2024.11 | 34 | 75 | 70 | 57 | 79 | 27 | 57 | |
| SVD-LLM V2Backbone Model=Llama-3.1-8B, Compression Rate=20%2024.11 | 33 | 79 | 70 | 58 | 77 | 36 | 59 | |
| FiPSBackbone Model=Llama-3.1-8B, Compression Rate=20%2024.11 | 33 | 79 | 72 | 59 | 78 | 38 | 60 | |
| SVD-LLM V2Backbone Model=Llama-7B, Compression Rate=20%2024.11 | 32 | 72 | 70 | 52 | 75 | 24 | 54 | |
| FiPSBackbone Model=Llama-7B, Compression Rate=20%2024.11 | 32 | 72 | 70 | 56 | 78 | 26 | 56 | |
| SVD-LLMBackbone Model=Llama-7B, Compression Rate=20%2024.11 | 31 | 71 | 68 | 49 | 71 | 22 | 52 | |
| ASVDBackbone Model=Llama-7B, Compression Rate=20%2024.11 | 29 | 53 | 64 | 41 | 68 | 17 | 45 | |
| SVD-LLMBackbone Model=Llama-3.1-8B, Compression Rate=20%2024.11 | 29 | 77 | 64 | 51 | 72 | 30 | 54 | |
| Basis SharingBackbone Model=Llama-7B, Compression Rate=20%2024.11 | 28 | 66 | 66 | 46 | 71 | 25 | 50 | |
| ASVDBackbone Model=Llama-3.1-8B, Compression Rate=20%2024.11 | 20 | 59 | 61 | 41 | 69 | 30 | 47 |