Commonsense Reasoning on 7 Reasoning Datasets (test)
60Overall Average AccuracyDense
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DenseMode=Zero-shot, Compression ratio=0%, Post-training=None2024.12 | 60 | 34 | 82 | 74 | 59 | 52 | 80 | 39 | 100 | |
| OriginalModel=LLaMA3.1-8B-Instruct, Compression ratio=0%2024.12 | 60 | — | — | — | — | — | — | — | — | |
| OriginalModel=Mistral-7B, Compression ratio=0%2024.12 | 59 | — | — | — | — | — | — | — | — | |
| OriginalModel=LLaMA2-13B, Compression ratio=0%2024.12 | 53 | — | — | — | — | — | — | — | — | |
| GRASPModel=LLaMA3.1-8B-Instruct, Compression ratio=20%, variant=Ours*2024.12 | 53 | — | — | — | — | — | — | — | — | |
| OriginalModel=LLaMA2-7B, Compression ratio=0%2024.12 | 52 | — | — | — | — | — | — | — | — | |
| GRASPModel=Mistral-7B, Compression ratio=20%, variant=Ours*2024.12 | 51 | — | — | — | — | — | — | — | — | |
| GRASPModel=LLaMA2-13B, Compression ratio=20%, variant=Ours*2024.12 | 48 | — | — | — | — | — | — | — | — | |
| GRASPMode=Zero-shot, Compression ratio=20%, Post-training=None2024.12 | 47 | 22 | 60 | 70 | 44 | 37 | 69 | 28 | 78.6 | |
| GRASPModel=LLaMA2-7B, Compression ratio=20%, variant=Ours*2024.12 | 47 | — | — | — | — | — | — | — | — | |
| SVD-LLMModel=LLaMA2-13B, Compression ratio=20%2024.12 | 47 | — | — | — | — | — | — | — | — | |
| GRASPModel=LLaMA3.1-8B-Instruct, Compression ratio=20%, post-training=none2024.12 | 47 | — | — | — | — | — | — | — | — | |
| ASVDModel=LLaMA2-13B, Compression ratio=20%2024.12 | 46 | — | — | — | — | — | — | — | — | |
| GRASPModel=Mistral-7B, Compression ratio=20%, post-training=none2024.12 | 45 | — | — | — | — | — | — | — | — | |
| ShortGPTMode=Zero-shot, Compression ratio=20%, Post-training=None2024.12 | 44 | 21 | 57 | 66 | 42 | 32 | 67 | 26 | 74.1 | |
| ASVDModel=LLaMA2-7B, Compression ratio=20%2024.12 | 44 | — | — | — | — | — | — | — | — | |
| SVD-LLMModel=LLaMA2-7B, Compression ratio=20%2024.12 | 44 | — | — | — | — | — | — | — | — | |
| GRASPModel=LLaMA2-7B, Compression ratio=20%, post-training=none2024.12 | 44 | — | — | — | — | — | — | — | — | |
| GRASPModel=LLaMA2-13B, Compression ratio=20%, post-training=none2024.12 | 44 | — | — | — | — | — | — | — | — | |
| LaCoMode=Zero-shot, Compression ratio=20%, Post-training=None2024.12 | 42 | 26 | 49 | 65 | 33 | 30 | 65 | 30 | 70.9 | |
| SVD-LLMModel=Mistral-7B, Compression ratio=20%2024.12 | 41 | — | — | — | — | — | — | — | — | |
| ASVDModel=Mistral-7B, Compression ratio=20%2024.12 | 40 | — | — | — | — | — | — | — | — | |
| SliceGPTMode=Zero-shot, Compression ratio=20%, Post-training=None2024.12 | 35 | 15 | 43 | 51 | 30 | 23 | 58 | 22 | 57.7 | |
| FWSVDModel=Mistral-7B, Compression ratio=20%2024.12 | 32 | — | — | — | — | — | — | — | — | |
| FWSVDModel=LLaMA2-7B, Compression ratio=20%2024.12 | 32 | — | — | — | — | — | — | — | — | |
| FWSVDModel=LLaMA2-13B, Compression ratio=20%2024.12 | 31 | — | — | — | — | — | — | — | — | |
| ASVDModel=LLaMA3.1-8B-Instruct, Compression ratio=20%2024.12 | 30 | — | — | — | — | — | — | — | — | |
| FWSVDModel=LLaMA3.1-8B-Instruct, Compression ratio=20%2024.12 | 29 | — | — | — | — | — | — | — | — |