Commonsense Reasoning on Commonsense Reasoning Suite
35OpenBookQA AccuracyDense
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DenseRatio=1.0, zero-shot evaluation=true2026.04 | 35 | 79 | 48 | 72 | 79 | 32 | 60 | 58 | — | |
| DenseCompression Ratio=1.0, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 34 | 75 | 42 | 69 | 79 | 27 | 57 | 55 | — | |
| DenseRatio=1.0, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 34 | 82 | 52 | 74 | 80 | 40 | 60 | 60 | — | |
| AA-SVDRatio=0.8, zero-shot evaluation=true2026.04 | 33 | 73 | 40 | 69 | 74 | 29 | 52 | 53 | 8.9 | |
| AA-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 31 | 71 | 36 | 66 | 72 | 25 | 48 | 50 | 8.9 | |
| AA-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, Weight Remapping=Dobi-SVD-style2026.04 | 30 | 74 | 41 | 69 | 77 | 26 | 53 | 53 | 3.4 | |
| SVD-LLMRatio=0.8, zero-shot evaluation=true2026.04 | 29 | 67 | 33 | 68 | 71 | 26 | 44 | 48 | 16.5 | |
| SAES-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 29 | 68 | 36 | 65 | 75 | 25 | 45 | 49 | 10.4 | |
| Dobi-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform, Weight Remapping=Dobi-SVD-style2026.04 | 28 | 65 | 32 | 62 | 72 | 25 | 45 | 47 | 14.1 | |
| AA-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, Weight Remapping=Dobi-SVD-style2026.04 | 28 | 71 | 37 | 65 | 73 | 26 | 49 | 50 | 8.9 | |
| Dip-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04 | 27 | 63 | 33 | 64 | 71 | 24 | 45 | 47 | 14.6 | |
| Dobi-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform, Weight Remapping=Dobi-SVD-style2026.04 | 27 | 65 | 37 | 68 | 77 | 27 | 54 | 51 | 7.3 | |
| DenseRatio=1.0, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04 | 26 | 65 | 31 | 61 | 74 | 29 | 48 | 48 | — | |
| Dobi-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04 | 26 | 59 | 31 | 66 | 70 | 23 | 44 | 46 | 16.7 | |
| AA-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 26 | 62 | 31 | 62 | 65 | 23 | 41 | 44 | 19.1 | |
| AA-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true, Weight Remapping=Dobi-SVD-style2026.04 | 26 | 58 | 31 | 62 | 64 | 23 | 41 | 44 | 20.4 | |
| AA-SVDRatio=0.8, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 26 | 70 | 37 | 69 | 72 | 30 | 46 | 50 | 17.1 | |
| AA-SVDRatio=0.6, zero-shot evaluation=true2026.04 | 25 | 64 | 32 | 63 | 68 | 26 | 41 | 46 | 21.2 | |
| ASVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 25 | 53 | 27 | 64 | 68 | 24 | 41 | 43 | 21.1 | |
| SAES-SVDRatio=0.8, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 25 | 59 | 28 | 66 | 67 | 27 | 39 | 44 | 26.3 | |
| SVD-LLMRatio=0.8, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 24 | 59 | 30 | 64 | 66 | 26 | 37 | 44 | 27.5 | |
| AA-SVDRatio=0.4, zero-shot evaluation=true2026.04 | 23 | 45 | 24 | 58 | 60 | 24 | 35 | 38 | 33.6 | |
| SAES-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 23 | 50 | 29 | 62 | 65 | 23 | 36 | 41 | 24.8 | |
| Dobi-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform, Weight Remapping=Dobi-SVD-style2026.04 | 23 | 52 | 24 | 56 | 65 | 23 | 38 | 40 | 26.6 | |
| AA-SVDRatio=0.6, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 23 | 54 | 27 | 61 | 64 | 24 | 37 | 41 | 31.3 | |
| SVD-LLMRatio=0.6, zero-shot evaluation=true2026.04 | 22 | 47 | 22 | 61 | 60 | 23 | 33 | 38 | 33.8 | |
| SVD-LLMCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, LoRA fine-tuning=true2026.04 | 22 | 58 | 29 | 63 | 69 | 24 | 43 | 44 | 19.6 | |
| Dobi-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04 | 22 | 41 | 27 | 58 | 61 | 23 | 34 | 38 | 30.5 | |
| Dip-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04 | 22 | 50 | 30 | 61 | 64 | 22 | 36 | 41 | 25.6 | |
| AA-SVDRatio=0.8, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04 | 20 | 51 | 23 | 55 | 64 | 23 | 36 | 39 | 18.6 | |
| AA-SVDRatio=0.6, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04 | 19 | 42 | 22 | 53 | 58 | 23 | 30 | 35 | 26.1 | |
| SVD-LLMCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, LoRA fine-tuning=true2026.04 | 19 | 42 | 25 | 58 | 60 | 21 | 33 | 37 | 32.6 | |
| AA-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 19 | 44 | 23 | 55 | 60 | 23 | 32 | 37 | 33.2 | |
| AA-SVDRatio=0.4, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 18 | 38 | 20 | 52 | 58 | 22 | 30 | 34 | 43.6 | |
| AA-SVDRatio=0.4, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04 | 16 | 35 | 20 | 52 | 56 | 21 | 27 | 32 | 32.1 | |
| SAES-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 16 | 33 | 25 | 52 | 54 | 23 | 30 | 33 | 39.2 | |
| SAES-SVDRatio=0.6, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 16 | 34 | 20 | 55 | 55 | 22 | 30 | 33 | 45 | |
| Dobi-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04 | 15 | 31 | 20 | 52 | 54 | 22 | 28 | 32 | 42 | |
| SVD-LLMRatio=0.4, zero-shot evaluation=true2026.04 | 14 | 31 | 20 | 52 | 54 | 21 | 27 | 31 | 45.9 | |
| SVD-LLMRatio=0.8, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04 | 14 | 37 | 19 | 51 | 56 | 21 | 28 | 32 | 32.3 | |
| SVD-LLMCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true, LoRA fine-tuning=true2026.04 | 14 | 28 | 22 | 50 | 55 | 21 | 27 | 31 | 43.3 | |
| SVD-LLMRatio=0.6, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 14 | 32 | 19 | 52 | 55 | 21 | 28 | 32 | 47.6 | |
| SVD-LLMRatio=0.4, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04 | 13 | 26 | 21 | 51 | 53 | 20 | 26 | 30 | 37.1 | |
| ASVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 13 | 28 | 22 | 48 | 55 | 19 | 26 | 30 | 44.9 | |
| SAES-SVDRatio=0.4, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 13 | 29 | 22 | 53 | 54 | 23 | 28 | 32 | 47.4 | |
| SVD-LLMRatio=0.6, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04 | 12 | 27 | 20 | 51 | 52 | 20 | 26 | 30 | 37.7 | |
| ASVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04 | 12 | 26 | 21 | 49 | 53 | 18 | 26 | 29 | 46.5 | |
| SVD-LLMRatio=0.4, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 12 | 27 | 20 | 51 | 52 | 20 | 26 | 30 | 50.7 |