List your company
ResearchFormerly WizWand

Commonsense Reasoning on Commonsense Reasoning Suite

35OpenBookQA AccuracyDense
11.0817.2923.529.71Apr 2, 2026
Updated 7mo ago

Evaluation results

MethodLinks
DenseRatio=1.0, zero-shot evaluation=true2026.043579487279326058—
DenseCompression Ratio=1.0, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.043475426979275755—
DenseRatio=1.0, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.043482527480406060—
AA-SVDRatio=0.8, zero-shot evaluation=true2026.0433734069742952538.9
AA-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.0431713666722548508.9
AA-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, Weight Remapping=Dobi-SVD-style2026.0430744169772653533.4
SVD-LLMRatio=0.8, zero-shot evaluation=true2026.04296733687126444816.5
SAES-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04296836657525454910.4
Dobi-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform, Weight Remapping=Dobi-SVD-style2026.04286532627225454714.1
AA-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, Weight Remapping=Dobi-SVD-style2026.0428713765732649508.9
Dip-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04276333647124454714.6
Dobi-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform, Weight Remapping=Dobi-SVD-style2026.0427653768772754517.3
DenseRatio=1.0, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.042665316174294848—
Dobi-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04265931667023444616.7
AA-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04266231626523414419.1
AA-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true, Weight Remapping=Dobi-SVD-style2026.04265831626423414420.4
AA-SVDRatio=0.8, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04267037697230465017.1
AA-SVDRatio=0.6, zero-shot evaluation=true2026.04256432636826414621.2
ASVDCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04255327646824414321.1
SAES-SVDRatio=0.8, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04255928666727394426.3
SVD-LLMRatio=0.8, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04245930646626374427.5
AA-SVDRatio=0.4, zero-shot evaluation=true2026.04234524586024353833.6
SAES-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04235029626523364124.8
Dobi-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform, Weight Remapping=Dobi-SVD-style2026.04235224566523384026.6
AA-SVDRatio=0.6, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04235427616424374131.3
SVD-LLMRatio=0.6, zero-shot evaluation=true2026.04224722616023333833.8
SVD-LLMCompression Ratio=0.8, Backbone=LLaMA-7B, Zero-shot evaluation=true, LoRA fine-tuning=true2026.04225829636924434419.6
Dobi-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04224127586123343830.5
Dip-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04225030616422364125.6
AA-SVDRatio=0.8, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04205123556423363918.6
AA-SVDRatio=0.6, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04194222535823303526.1
SVD-LLMCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true, LoRA fine-tuning=true2026.04194225586021333732.6
AA-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04194423556023323733.2
AA-SVDRatio=0.4, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04183820525822303443.6
AA-SVDRatio=0.4, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04163520525621273232.1
SAES-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04163325525423303339.2
SAES-SVDRatio=0.6, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04163420555522303345
Dobi-SVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true, Rank Allocation=Dynamic/non-uniform2026.04153120525422283242
SVD-LLMRatio=0.4, zero-shot evaluation=true2026.04143120525421273145.9
SVD-LLMRatio=0.8, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04143719515621283232.3
SVD-LLMCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true, LoRA fine-tuning=true2026.04142822505521273143.3
SVD-LLMRatio=0.6, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04143219525521283247.6
SVD-LLMRatio=0.4, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04132621515320263037.1
ASVDCompression Ratio=0.6, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04132822485519263044.9
SAES-SVDRatio=0.4, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04132922535423283247.4
SVD-LLMRatio=0.6, Backbone=LLaMA-3-1B, Evaluation Protocol=zero-shot2026.04122720515220263037.7
ASVDCompression Ratio=0.4, Backbone=LLaMA-7B, Zero-shot evaluation=true2026.04122621495318262946.5
SVD-LLMRatio=0.4, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04122720515220263050.7