Zero-shot Reasoning on (ARC-C, BoolQ, HellaSwag, MathQA, MMLU, Race)
50.43ARC-C AccuracyBase
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BaseModel=Mistral-7B, Compression=None, Evaluation Protocol=Zero-shot2025.10 | 50.43 | 83.61 | 81.08 | 35.71 | 59.64 | 40.86 | 58.56 | |
| BaseModel=LLaMA-2-7B, Compression=None, Evaluation Protocol=Zero-shot2025.10 | 43.34 | 77.71 | 76 | 28.07 | 41.82 | 39.52 | 51.08 | |
| PGSVDModel=Mistral-7B, Compression=20%, Evaluation Protocol=Zero-shot2025.10 | 37.71 | 66.21 | 62.26 | 29.25 | 28.5 | 36.94 | 43.48 | |
| PGSVDModel=LLaMA-2-7B, Compression=20%, Evaluation Protocol=Zero-shot2025.10 | 36.52 | 67.43 | 60.96 | 25.76 | 27.48 | 35.02 | 42.2 | |
| SVD-ALSModel=Mistral-7B, Compression=20%, Evaluation Protocol=Zero-shot2025.10 | 36.35 | 65.81 | 59.74 | 28.38 | 27.99 | 35.5 | 42.3 | |
| SVD-LLMModel=Mistral-7B, Compression=20%, Evaluation Protocol=Zero-shot2025.10 | 35.15 | 67.74 | 59.6 | 28.54 | 28.53 | 35.5 | 42.51 | |
| SVD-LLMModel=LLaMA-2-7B, Compression=20%, Evaluation Protocol=Zero-shot2025.10 | 34.39 | 63.09 | 59.19 | 25.33 | 26.2 | 34.55 | 40.46 | |
| SVD-ALSModel=LLaMA-2-7B, Compression=20%, Evaluation Protocol=Zero-shot2025.10 | 33.7 | 68.01 | 59.55 | 24.96 | 26.64 | 33.97 | 41.14 | |
| PGSVDModel=LLaMA-2-7B, Compression=40%, Evaluation Protocol=Zero-shot2025.10 | 23.63 | 56.12 | 43.42 | 22.65 | 23.29 | 28.61 | 32.62 | |
| SVD-LLMModel=LLaMA-2-7B, Compression=40%, Evaluation Protocol=Zero-shot2025.10 | 22.95 | 43.21 | 41.73 | 22.24 | 23.34 | 28.8 | 30.83 | |
| SVD-ALSModel=LLaMA-2-7B, Compression=40%, Evaluation Protocol=Zero-shot2025.10 | 22.87 | 42.57 | 41.74 | 22.38 | 23.54 | 27.85 | 30.16 | |
| SVD-LLMModel=Mistral-7B, Compression=40%, Evaluation Protocol=Zero-shot2025.10 | 21.67 | 38.62 | 37.87 | 22.18 | 23.38 | 27.08 | 28.47 | |
| PGSVDModel=Mistral-7B, Compression=40%, Evaluation Protocol=Zero-shot2025.10 | 21.42 | 39.02 | 39.12 | 22.35 | 23.44 | 27.46 | 28.8 | |
| SVD-ALSModel=Mistral-7B, Compression=40%, Evaluation Protocol=Zero-shot2025.10 | 20.82 | 38.41 | 37.87 | 22.75 | 23.28 | 27.18 | 28.39 |