List your company
ResearchFormerly WizWand

Commonsense Reasoning on Six common sense reasoning tasks (Zero-shot)

77.51Average Accuracy (Zero-shot)FPTQuant
27.236440.288253.3466.3918Jun 5, 2025Aug 6, 2025Oct 7, 2025Dec 9, 2025Feb 9, 2026Apr 12, 2026Jun 14, 2026
Updated 3mo ago

Evaluation results

MethodLinks
FPTQuantBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.0677.51
QuaRot-optBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.0676.24
FlatQuantBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.0676.22
OSTQuantBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.0675.96
SpinQuantBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.0675.51
FP16Bits (W-A-KV)=16-16-16, Model=Q2.5-32B, Zero-shot=true2025.0675.29
Full Precision BaselinePrecision=W16A16, Model=Qwen3 32B, Zero-shot=true2026.0675.2
RTN-optBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.0675.15
FP16Bits (W-A-KV)=16-16-16, Model=M-8B it, Zero-shot=true2025.0674.37
Full Precision BaselinePrecision=W16A16, Model=Qwen3 14B, Zero-shot=true2026.0674.2
MosaicQuantPrecision=W4A8, Model=Qwen3 32B, Zero-shot=true2026.0674.2
FPTQuantBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.0673.59
Full Precision BaselinePrecision=W16A16, Model=LLaMA3 8B, Zero-shot=true2026.0673.5
QuaRot-optBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.0673.41
FlatQuantBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.0673.41
MosaicQuantPrecision=W4A8, Model=Qwen3 14B, Zero-shot=true2026.0673.4
FP16Bits (W-A-KV)=16-16-16, Model=L3-8B, Zero-shot=true2025.0673.33
MosaicQuantPrecision=W4A4, Model=Qwen3 32B, Zero-shot=true2026.0673.3
FlatQuantBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.0673.05
FPTQuantBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.0672.92
MosaicQuantPrecision=W4A4, Model=Qwen3 14B, Zero-shot=true2026.0672.9
OSTQuantBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.0672.87
FPTQuantBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.0672.72
MosaicQuantPrecision=W4A8, Model=LLaMA3 8B, Zero-shot=true2026.0672.7
SpinQuantPrecision=W4A4, Model=Qwen3 32B, Zero-shot=true2026.0672.7
SpinQuantBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.0672.48
AtomPrecision=W4A4, Model=Qwen3 32B, Zero-shot=true2026.0672.2
FlatQuantBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.0672.11
ResQPrecision=W4A4, Model=Qwen3 32B, Zero-shot=true2026.0672
Full Precision BaselinePrecision=W16A16, Model=Qwen3 8B, Zero-shot=true2026.0671.6
SpinQuantBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.0671.6
OSTQuantBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.0671.46
MosaicQuantPrecision=W4A4, Model=LLaMA3 8B, Zero-shot=true2026.0671.2
MosaicQuantPrecision=W4A8, Model=Qwen3 8B, Zero-shot=true2026.0670.6
RTNBits (W-A-KV)=4-8-8, Model=Q2.5-32B, Zero-shot=true2025.0670.28
FP16Bits (W-A-KV)=16-16-16, Model=L2-7B, Zero-shot=true2025.0669.79
Full Precision BaselinePrecision=W16A16, Model=Qwen3 4B, Zero-shot=true2026.0669.4
SpinQuantBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.0669.12
MosaicQuantPrecision=W4A8, Model=Qwen3 4B, Zero-shot=true2026.0668.7
MosaicQuantPrecision=W4A4, Model=Qwen3 8B, Zero-shot=true2026.0668.6
QuaRot-optBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.0668.01
QuaRot-optBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.0667.6
RTN-optBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.0667.35
OSTQuantBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.0666.84
MosaicQuantPrecision=W4A4, Model=Qwen3 4B, Zero-shot=true2026.0666.3
Full Precision BaselinePrecision=W16A16, Model=LLaMA3.2 3B, Zero-shot=true2026.0666.2
SpinQuantBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.0666.01
FPTQuantBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.0665.96
FP16Bits (W-A-KV)=16-16-16, Model=L3.2-3B it, Zero-shot=true2025.0665.63
MosaicQuantPrecision=W4A8, Model=LLaMA3.2 3B, Zero-shot=true2026.0665.3
FlatQuantBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.0665.04
MosaicQuantPrecision=W4A4, Model=LLaMA3.2 3B, Zero-shot=true2026.0664.9
QuaRot-optBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.0664.68
FPTQuantBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.0664
FPTQuantBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.0663.69
FlatQuantBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.0663.51
QuaRot-optBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.0663.43
SpinQuantBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.0663.28
QuaRot-optBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.0663.12
AWQPrecision=W4A16, Model=LLaMA3.2 3B, Zero-shot=true2026.0663.1
OSTQuantBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.0662.48
FlatQuantBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.0662.07
OSTQuantBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.0661.85
GPTQPrecision=W4A16, Model=LLaMA3.2 3B, Zero-shot=true2026.0661.7
FlatQuantBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.0661.43
RTN-optBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.0661.09
FlatQuantBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.0661
RTNPrecision=W4A16, Model=LLaMA3.2 3B, Zero-shot=true2026.0660.9
SpinQuantBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.0660.6
FPTQuantBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.0659.46
RTN-optBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.0656.93
OSTQuantBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.0656.69
SpinQuantBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.0654.88
SpinQuantBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.0654.58
QuaRot-optBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.0654.38
FPTQuantBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.0652.96
OSTQuantBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.0652.43
RTN-optBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.0651.97
OSTQuantBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.0650.88
RTNBits (W-A-KV)=4-8-8, Model=L2-7B, Zero-shot=true2025.0647.75
RTNBits (W-A-KV)=4-8-8, Model=L3.2-3B it, Zero-shot=true2025.0647.27
RTNBits (W-A-KV)=4-8-8, Model=L3-8B, Zero-shot=true2025.0645
QuaRot-optBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.0642.76
RTNBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.0639.13
RTNBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.0637.67
RTN-optBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.0636.92
RTN-optBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.0631.16
RTNBits (W-A-KV)=4-8-8, Model=M-8B it, Zero-shot=true2025.0631.02
OSTQuantBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.0630.75
QuaRot-optBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.0630.21
SpinQuantBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.0630.17
RTN-optBits (W-A-KV)=4-4-4, Model=L3-8B, Zero-shot=true2025.0630.04
FlatQuantBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.0629.9
RTNBits (W-A-KV)=4-4-4, Model=Q2.5-32B, Zero-shot=true2025.0629.83
FPTQuantBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.0629.76
RTN-optBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.0629.63
RTN-optBits (W-A-KV)=4-4-4, Model=L2-7B, Zero-shot=true2025.0629.54
RTNBits (W-A-KV)=4-4-4, Model=M-8B it, Zero-shot=true2025.0629.33
RTNBits (W-A-KV)=4-4-4, Model=L3.2-3B it, Zero-shot=true2025.0629.17