List your company
ResearchFormerly WizWand

Common Sense Reasoning on Six common sense reasoning tasks

69.46AccuracyFP
0.341618.285836.2354.1742Mar 4, 2026Mar 24, 2026Apr 13, 2026May 3, 2026May 23, 2026Jun 12, 2026Jul 3, 2026
Updated 3mo ago

Evaluation results

MethodLinks
FPBackbone=Llama 2 7B, Evaluation Protocol=Zero-shot2026.0369.46
GPTQ FlatQuantBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=FlatQuant, Train=true, Evaluation Protocol=Zero-shot2026.0367.58
CAT (block)Backbone=Llama 2 7B, Weight quantization=RTN, Transform method=CAT (block), Train=true, Evaluation Protocol=Zero-shot2026.0367.34
CAT (block)Backbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=CAT (block), Train=false, Evaluation Protocol=Zero-shot2026.0367.25
CAT (block)Backbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=CAT (block), Train=true, Evaluation Protocol=Zero-shot2026.0367.17
RTN FlatQuantBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=FlatQuant, Train=true, Evaluation Protocol=Zero-shot2026.0366.67
CAT (block)Backbone=Llama 2 7B, Weight quantization=RTN, Transform method=CAT (block), Train=false, Evaluation Protocol=Zero-shot2026.0366.22
GPTQ SpinQuantBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=SpinQuant, Train=true, Evaluation Protocol=Zero-shot2026.0366.12
GPTQ QuaRotBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=QuaRot, Train=false, Evaluation Protocol=Zero-shot2026.0366.06
RTN SpinQuantBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=SpinQuant, Train=true, Evaluation Protocol=Zero-shot2026.0363.64
TOGA-qPrecision=W(3.2)A(3.2), Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0662.6
OriginalBackbone=Mistral-7B, Compression Ratio=0%2026.0761
ResQPrecision=W(3.2)A(3.2), Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0658.43
RTN QuaRotBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=QuaRot, Train=false, Evaluation Protocol=Zero-shot2026.0357.54
OriginalBackbone=LLaMA 2-7B, Compression Ratio=0%2026.0757
TOGA-qPrecision=W(3.2)A(3.2), Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0656.03
OriginalBackbone=Vicuna-7B, Compression Ratio=0%2026.0756
LACE-SVDBackbone=LLaMA 2-7B, Compression Ratio=20%2026.0755
TOGA-qPrecision=W(3.2)A(3.2), Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0654.2
LACE-SVDBackbone=Vicuna-7B, Compression Ratio=20%2026.0754
SVD-LLMBackbone=LLaMA 2-7B, Compression Ratio=20%2026.0753
ResQPrecision=W(3.2)A(3.2), Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0652.63
OriginalBackbone=OPT-6.7B, Compression Ratio=0%2026.0752
AtomPrecision=W(3.2)A(3.2), Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0651.71
SVD-LLMBackbone=Vicuna-7B, Compression Ratio=20%2026.0751
TOGA-qPrecision=W(3.2)A(3.2), Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0650.81
LACE-SVDBackbone=Mistral-7B, Compression Ratio=20%2026.0750
AtomPrecision=W(3.2)A(3.2), Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0648.13
TOGA-qPrecision=W(1.6)A16, Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0647.58
TOGA-qPrecision=W(1.6)A16, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0646.8
ResQPrecision=W(3.2)A(3.2), Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0646.33
GPTQ SmoothQuantBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=SmoothQuant, Train=false, Evaluation Protocol=Zero-shot2026.0344.62
SilM-LLMPrecision=W2A16, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0643.94
PTQ-1.61†Precision=W(1.6)A16, Backbone=Qwen3-8B, Calibration Pre-processing=RedPajama dataset, Zero-shot Evaluation Protocol=Average of six tasks2026.0643.53
SilM-LLMPrecision=W2A16, Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0643
LACE-SVDBackbone=OPT-6.7B, Compression Ratio=20%2026.0743
ResQPrecision=W(3.2)A(3.2), Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0642.45
TOGA-qPrecision=W(1.6)A16, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0642.07
SVD-LLMBackbone=Mistral-7B, Compression Ratio=20%2026.0742
PTQ-1.61†Precision=W(1.6)A16, Backbone=Llama-2-7B, Calibration Pre-processing=RedPajama dataset, Zero-shot Evaluation Protocol=Average of six tasks2026.0641.43
TOGA-qPrecision=W(1.6)A16, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0641.33
BiLLMPrecision=W(2.1)A16, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0641.16
SVD-LLMBackbone=OPT-6.7B, Compression Ratio=20%2026.0741
SilM-LLMPrecision=W2A16, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0640.03
PTQ-1.61†Precision=W(1.6)A16, Backbone=Mistral-7B, Calibration Pre-processing=RedPajama dataset, Zero-shot Evaluation Protocol=Average of six tasks2026.0639.77
AtomPrecision=W(3.2)A(3.2), Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0639.57
BiLLMPrecision=W(2.1)A16, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0639.16
BiLLMPrecision=W(2.1)A16, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0639.11
AtomPrecision=W(3.2)A(3.2), Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0638.36
BiLLMPrecision=W(2.1)A16, Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0637.87
PTQ-1.61Precision=W(1.6)A16, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0637.15
SilM-LLMPrecision=W2A16, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0636.12
ASVDBackbone=LLaMA 2-7B, Compression Ratio=20%2026.0736
PTQ-1.61†Precision=W(1.6)A16, Backbone=Llama-3-8B, Calibration Pre-processing=RedPajama dataset, Zero-shot Evaluation Protocol=Average of six tasks2026.0635.59
RTN SmoothQuantBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=SmoothQuant, Train=false, Evaluation Protocol=Zero-shot2026.0334.49
SpinQuantPrecision=W3A3, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0634.26
PTQ-1.61Precision=W(1.6)A16, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0634.22
PTQ-1.61Precision=W(1.6)A16, Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0634.02
SpinQuantPrecision=W3A3, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0633.96
PTQ-1.61Precision=W(1.6)A16, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.0633.76
SpinQuantPrecision=W3A3, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.0633.06
ASVDBackbone=Vicuna-7B, Compression Ratio=20%2026.0733
ASVDBackbone=OPT-6.7B, Compression Ratio=20%2026.0732
ASVDBackbone=Mistral-7B, Compression Ratio=20%2026.0732
GPTQ NoneBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=None, Train=false, Evaluation Protocol=Zero-shot2026.0331.88
RTN NoneBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=None, Train=false, Evaluation Protocol=Zero-shot2026.0331.51
FWSVDBackbone=LLaMA 2-7B, Compression Ratio=20%2026.0712
SVDBackbone=LLaMA 2-7B, Compression Ratio=20%2026.079
FWSVDBackbone=Vicuna-7B, Compression Ratio=20%2026.079
FWSVDBackbone=Mistral-7B, Compression Ratio=20%2026.078
FWSVDBackbone=OPT-6.7B, Compression Ratio=20%2026.076
SVDBackbone=Vicuna-7B, Compression Ratio=20%2026.075
SVDBackbone=OPT-6.7B, Compression Ratio=20%2026.073
SVDBackbone=Mistral-7B, Compression Ratio=20%2026.073