Reasoning on ARC-E, CSQA, Lambada, PIQA, Wino
80.85ARC-E AccuracyBase
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BaseModel=Mistral-7B, Compression=Base2025.10 | 80.85 | 56.27 | 69.49 | 80.63 | 74.19 | 72.29 | |
| BaseModel=LLaMA-2-13B, Compression=Base2025.10 | 79.46 | 46.6 | 70.33 | 79.11 | 72.3 | 69.56 | |
| BaseModel=LLaMA-2-7B, Compression=Base2025.10 | 76.3 | 33.01 | 68.25 | 78.07 | 69.06 | 64.94 | |
| PGSVDModel=LLaMA-2-13B, Compression=20%2025.10 | 73.36 | 25.14 | 60.78 | 73.23 | 69.38 | 60.38 | |
| PGSVDModel=Mistral-7B, Compression=20%2025.10 | 72.31 | 21.05 | 52.8 | 73.07 | 66.46 | 57.14 | |
| SVD-LLMModel=LLaMA-2-13B, Compression=20%2025.10 | 71 | 25.23 | 57.54 | 72.91 | 67.17 | 58.77 | |
| SVD-ALSModel=Mistral-7B, Compression=20%2025.10 | 70.88 | 21.95 | 50.38 | 71.65 | 65.82 | 56.14 | |
| PGSVDModel=LLaMA-2-7B, Compression=20%2025.10 | 70.75 | 20.8 | 53.02 | 71.27 | 64.56 | 56.08 | |
| SVD-LLMModel=Mistral-7B, Compression=20%2025.10 | 69.87 | 22.03 | 50.32 | 71.6 | 65.19 | 55.8 | |
| SVD-ALSModel=LLaMA-2-7B, Compression=20%2025.10 | 68.73 | 20.8 | 47.18 | 70.57 | 64.8 | 54.42 | |
| SVD-LLMModel=LLaMA-2-7B, Compression=20%2025.10 | 68.56 | 19.82 | 46.61 | 70.35 | 64.33 | 53.93 | |
| SVD-ALSModel=LLaMA-2-13B, Compression=20%2025.10 | 66.12 | 20.31 | 45.97 | 69.26 | 63.46 | 53.02 | |
| PGSVDModel=LLaMA-2-13B, Compression=40%2025.10 | 59.34 | 19.98 | 31.71 | 64.15 | 62.67 | 47.57 | |
| SVD-ALSModel=LLaMA-2-13B, Compression=40%2025.10 | 57.87 | 22.11 | 26.86 | 63.6 | 62.12 | 46.51 | |
| SVD-LLMModel=LLaMA-2-13B, Compression=40%2025.10 | 56.61 | 19.49 | 26.47 | 63.22 | 60.85 | 45.33 | |
| PGSVDModel=LLaMA-2-7B, Compression=40%2025.10 | 54.76 | 19.25 | 21.66 | 62.89 | 59.75 | 43.66 | |
| SVD-ALSModel=LLaMA-2-7B, Compression=40%2025.10 | 50.88 | 19.16 | 16.59 | 61.21 | 58.88 | 41.35 | |
| SVD-LLMModel=LLaMA-2-7B, Compression=40%2025.10 | 50.21 | 19.49 | 16.17 | 60.94 | 58.64 | 41.09 | |
| PGSVDModel=Mistral-7B, Compression=40%2025.10 | 49.24 | 20.23 | 16.48 | 60.45 | 58.48 | 40.98 | |
| SVD-ALSModel=Mistral-7B, Compression=40%2025.10 | 47.18 | 19.49 | 16.13 | 58.76 | 57.62 | 39.84 | |
| SVD-LLMModel=Mistral-7B, Compression=40%2025.10 | 46.3 | 19.41 | 15.93 | 59.25 | 57.14 | 39.61 |