Zero-shot Question Answering on (PiQA, ARC-E, ARC-C, HellaSwag, Winogrande, BoolQ, OBQA, SiQA)
80.63PiQA AccuracyFull Rank
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Full RankModel=LLaMA3-8B, Compression Ratio=0%, Evaluation Protocol=Zero-shot2026.04 | 80.63 | 77.74 | 53.5 | 79.12 | 73.32 | 81.19 | 44.8 | 33.01 | 65.41 | |
| Full RankModel=LLaMA2-7B, Compression Ratio=0%, Evaluation Protocol=Zero-shot2026.04 | 79.05 | 74.54 | 46.25 | 75.99 | 68.9 | 77.71 | 44.2 | 32.91 | 62.44 | |
| OBD-LLMModel=LLaMA2-7B, Compression Ratio=20%, Evaluation Protocol=Zero-shot2026.04 | 74.37 | 58.67 | 34.9 | 61.34 | 66.22 | 70.03 | 36.2 | 34.14 | 54.48 | |
| OBD-LLMModel=LLaMA3-8B, Compression Ratio=20%, Evaluation Protocol=Zero-shot2026.04 | 73.72 | 59.47 | 34.9 | 59.47 | 67.96 | 68.56 | 36.8 | 32.96 | 54.23 | |
| SVD-LLMModel=LLaMA2-7B, Compression Ratio=20%, Evaluation Protocol=Zero-shot2026.04 | 72.63 | 52.61 | 32.25 | 58.68 | 64.17 | 64.31 | 36.8 | 33.62 | 51.88 | |
| SVD-LLMModel=LLaMA3-8B, Compression Ratio=20%, Evaluation Protocol=Zero-shot2026.04 | 72.25 | 54.84 | 32.68 | 57.51 | 66.06 | 68.29 | 37 | 32.91 | 52.69 | |
| ASVDModel=LLaMA2-7B, Compression Ratio=20%, Evaluation Protocol=Zero-shot2026.04 | 63.82 | 47.22 | 29.61 | 46.6 | 57.22 | 59.6 | 30.4 | 31.99 | 45.81 | |
| ASVDModel=LLaMA3-8B, Compression Ratio=20%, Evaluation Protocol=Zero-shot2026.04 | 63.38 | 43.86 | 27.65 | 41.95 | 56.75 | 66.45 | 32.4 | 33.42 | 45.73 | |
| FWSVDModel=LLaMA3-8B, Compression Ratio=20%, Evaluation Protocol=Zero-shot2026.04 | 51.58 | 25.21 | 27.73 | 26.22 | 50.12 | 47.4 | 29.2 | 33.57 | 36.38 | |
| FWSVDModel=LLaMA2-7B, Compression Ratio=20%, Evaluation Protocol=Zero-shot2026.04 | 51.58 | 25.21 | 27.73 | 26.22 | 50.12 | 47.4 | 29.2 | 33.57 | 36.38 |