Zero-shot Accuracy on Reasoning Suite (PIQA, Hella Swag, LAMBADA, ARC-e, ARC-c, SciQ, Race, MMLU)
80.7PIQALlama3 8B
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Llama3 8BBackbone=Llama3 8B, Compression Ratio=Uncompressed2025.09 | 80.7 | 79.1 | 75.6 | 77.7 | 53.5 | 93.9 | 40.3 | 62.2 | 70.4 | |
| Qwen3 14BBackbone=Qwen3 14B, Compression Ratio=Uncompressed2025.09 | 79.9 | 78.9 | 67.9 | 82.8 | 60.2 | 96.5 | 43.3 | 77.2 | 73.3 | |
| Qwen3 8BBackbone=Qwen3 8B, Compression Ratio=Uncompressed2025.09 | 77.7 | 74.9 | 64.1 | 80.7 | 56.7 | 95.7 | 40.9 | 73 | 70.5 | |
| CoSpaDiBackbone=Qwen3 14B, Compression Ratio=0.22025.09 | 77.3 | 71.7 | 72 | 73.3 | 51.3 | 91.9 | 43.1 | 65.8 | 68.3 | |
| CoSpaDiBackbone=Qwen3 8B, Compression Ratio=0.22025.09 | 76.5 | 68 | 65.6 | 72.2 | 48.9 | 93.2 | 40.7 | 60.8 | 65.7 | |
| SVD-LLMBackbone=Qwen3 14B, Compression Ratio=0.22025.09 | 76.2 | 67.6 | 69.1 | 69.8 | 46.8 | 91 | 42.9 | 62.5 | 65.8 | |
| CoSpaDiBackbone=Llama3 8B, Compression Ratio=0.22025.09 | 75.2 | 66.5 | 73.8 | 66.5 | 41.6 | 89.5 | 38.2 | 42.8 | 61.8 | |
| CoSpaDiBackbone=Qwen3 14B, Compression Ratio=0.32025.09 | 74.6 | 64.2 | 69 | 65.3 | 43.6 | 90.7 | 42.5 | 58 | 63.5 | |
| SVD-LLMBackbone=Qwen3 8B, Compression Ratio=0.22025.09 | 73.8 | 63.9 | 62.2 | 68.7 | 45.7 | 90.1 | 40.5 | 54.7 | 62.5 | |
| CoSpaDiBackbone=Qwen3 8B, Compression Ratio=0.32025.09 | 72.4 | 60.5 | 62.6 | 63.9 | 41.2 | 88.4 | 39.5 | 51.3 | 59.97 | |
| SVD-LLMBackbone=Qwen3 14B, Compression Ratio=0.32025.09 | 72 | 59.7 | 61.5 | 62.6 | 39.3 | 88.9 | 41.5 | 52 | 59.7 | |
| SVD-LLMBackbone=Llama3 8B, Compression Ratio=0.22025.09 | 71.1 | 58.4 | 59.3 | 55.5 | 34 | 86.4 | 35.5 | 32.6 | 54.1 | |
| CoSpaDiBackbone=Llama3 8B, Compression Ratio=0.32025.09 | 70.5 | 56.2 | 61.3 | 54.2 | 33.5 | 85.7 | 36.2 | 32.2 | 53.7 | |
| SVD-LLMBackbone=Qwen3 8B, Compression Ratio=0.32025.09 | 70.4 | 55.2 | 53.8 | 59.3 | 37.1 | 87.2 | 38.4 | 44.8 | 55.8 | |
| CoSpaDiBackbone=Qwen3 14B, Compression Ratio=0.42025.09 | 69.5 | 52.3 | 57.3 | 53.2 | 33 | 84.4 | 37.5 | 43.6 | 53.9 | |
| CoSpaDiBackbone=Qwen3 8B, Compression Ratio=0.42025.09 | 68.9 | 49 | 49.9 | 49.4 | 29.9 | 82 | 36.8 | 36.6 | 50.3 | |
| SVD-LLMBackbone=Qwen3 14B, Compression Ratio=0.42025.09 | 67.4 | 48.7 | 46 | 48.5 | 30.7 | 80.6 | 36.7 | 36.8 | 49.4 | |
| SVD-LLMBackbone=Qwen3 8B, Compression Ratio=0.42025.09 | 66.3 | 44.6 | 37.9 | 45 | 28.1 | 77.3 | 35.3 | 29.1 | 45.4 | |
| SVD-LLMBackbone=Llama3 8B, Compression Ratio=0.32025.09 | 65.8 | 46.4 | 38.1 | 41.9 | 27.7 | 70 | 31.8 | 27.2 | 43.6 | |
| CoSpaDiBackbone=Llama3 8B, Compression Ratio=0.42025.09 | 63.7 | 41.4 | 30.3 | 39.1 | 26.6 | 68.5 | 30.5 | 25.4 | 40.7 | |
| SVD-LLMBackbone=Llama3 8B, Compression Ratio=0.42025.09 | 60.3 | 34.5 | 11.4 | 32.4 | 24.5 | 44.2 | 25.7 | 23.1 | 32 |