List your company
ResearchFormerly WizWand

Zero-shot Common Sense Reasoning on Common Sense Reasoning

72.6Zero-shot AccuracyFP16
51.966457.323262.6868.0368Jun 5, 2025Jul 29, 2025Sep 22, 2025Nov 16, 2025Jan 9, 2026Mar 5, 2026Apr 29, 2026
Updated 3mo ago

Evaluation results

MethodLinks
FP16Family=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=16/16/162026.0472.6
baselineBackbone=LLaMA-2 70B, #Bits=FP16, #Eff. (w)=162026.0370.54
SERQ (RTN)Backbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.442026.0370.35
SERQ (GPTQ)Backbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.432026.0370.31
FP16Family=Qwen2.5, Model=Qwen2.5-7B, W/A/KV=16/16/162026.0470.13
CoQuantFamily=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=4.5/4.5/4.52026.0470.01
L2QERBackbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.552026.0369.78
SERQ (RTN)Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.652026.0369.68
ResQFamily=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=4.5/4.5/4.52026.0469.64
SERQ-MXFP4Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.792026.0369.58
SERQ (GPTQ)Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.642026.0369.38
L2QER-MXFP4Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.812026.0369.03
QuaRotFamily=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=4/4/42026.0468.68
CoQuantFamily=Qwen2.5, Model=Qwen2.5-7B, W/A/KV=4.5/4.5/4.52026.0467.44
baselineBackbone=LLaMA-3 8B, #Bits=FP16, #Eff. (w)=162026.0367.16
L2QERBackbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=4.552026.0366.92
FP16Backbone=LLaMA-2 13B2026.0366.53
ResQFamily=Qwen2.5, Model=Qwen2.5-7B, W/A/KV=4.5/4.5/4.52026.0466.46
SERQ (RTN)Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=6.712026.0366.4
SERQ (GPTQ)Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=6.522026.0366.23
DenseModel Size=Large, Model=Llama-7B, Compression Factor (CF)=2x, Evaluation Protocol=Zero-shot2025.1266.07
L2QERBackbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=7.162026.0365.68
FP16Model=Llama-3.2 3B-it, # Bits (W-A-KV)=16-16-162025.0665.63
FlatQuantBackbone=LLaMA-2 13B, Calib. Time=∼225m2026.0364.94
OstQuantBackbone=LLaMA-2 13B, Calib. Time=∼92m2026.0364.92
DuQuantBackbone=LLaMA-2 13B, Calib. Time=∼486m2026.0364.88
SERQBackbone=LLaMA-2 13B, Calib. Time=∼48m2026.0364.82
SpinQuantBackbone=LLaMA-2 13B, Calib. Time=∼721m2026.0364.8
FP16Family=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=16/16/162026.0464.18
FP16Backbone=LLaMA-2 7B2026.0364.09
baselineBackbone=LLaMA-2 7B, #Bits=FP16, #Eff. (w)=162026.0364.09
FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.0663.84
baselineBackbone=Qwen-2.5-3B, #Bits=FP16, #Eff. (w)=162026.0363.82
SERQ (RTN)Backbone=Qwen-2.5-3B, #Bits=W4A8, #Eff. (w)=4.242026.0363.78
SERQ (GPTQ)Backbone=Qwen-2.5-3B, #Bits=W4A8, #Eff. (w)=4.242026.0363.7
SERQ (RTN)Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.242026.0363.41
QUIKFamily=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=4.5/4.5/4.52026.0463.39
L2QERBackbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.352026.0363.35
L2QER-MXFP4Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.832026.0363.33
SERQ (GPTQ)Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.242026.0363.04
FP16Family=Qwen2.5, Model=Qwen2.5-1.5B, W/A/KV=16/16/162026.0462.97
FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.0662.95
SERQ-MXFP4Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.632026.0362.71
QuaRotModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.0662.71
FP16Backbone=LLaMA-3.2 3B2026.0362.66
baselineBackbone=LLaMA-3.2 3B, #Bits=FP16, #Eff. (w)=162026.0362.66
QuaRotBackbone=LLaMA-2 13B, Calib. Time=∼44m2026.0362.55
SERQ (RTN)Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=8.072026.0362.49
SERQ (GPTQ)Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.752026.0362.41
FlatQuantBackbone=LLaMA-2 7B, Calib. Time=∼131m2026.0362.11
SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.0661.96
SERQBackbone=LLaMA-2 7B, Calib. Time=∼23m2026.0361.87
SERQ (GPTQ)Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.0361.87
DuQuantBackbone=LLaMA-2 7B, Calib. Time=∼255m2026.0361.84
SERQ (RTN)Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.0361.77
OstQuantBackbone=LLaMA-2 7B, Calib. Time=∼72m2026.0361.72
SERQ (GPTQ)Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.242026.0361.54
SERQ (RTN)Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.242026.0361.27
SERQ-MXFP4Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.372026.0361.26
SERQ (RTN)Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.242026.0361.24
SpinQuantBackbone=LLaMA-2 7B, Calib. Time=∼598m2026.0361
L2QER-MXFP4Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.372026.0360.95
LLM.int4()Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=-†2026.0360.86
SERQ (GPTQ)Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.242026.0360.78
L2QER-MXFP4Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.372026.0360.56
SERQ-MXFP4Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.372026.0360.43
FlatQuantBackbone=LLaMA-3.2 3B, Calib. Time=∼76m2026.0360.41
L2QERBackbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.352026.0360.36
CoQuantFamily=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=4.5/4.5/4.52026.0460.23
FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.0660.07
ResQFamily=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=4.5/4.5/4.52026.0460.03
LLM.int4()Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=-†2026.0359.64
QuaRotBackbone=LLaMA-2 7B, Calib. Time=∼31m2026.0359.53
FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.0659.43
OstQuantBackbone=LLaMA-3.2 3B, Calib. Time=∼26m2026.0358.81
RTN-optModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.0658.65
SERQBackbone=LLaMA-3.2 3B, Calib. Time=∼15m2026.0358.5
SERQ (GPTQ)Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.242026.0358.5
L2QER-MXFP4Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.372026.0358.41
SERQ (RTN)Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.242026.0358.4
DuQuantBackbone=LLaMA-3.2 3B, Calib. Time=∼140m2026.0358.31
L2QERBackbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.0357.67
CoQuantFamily=Qwen2.5, Model=Qwen2.5-1.5B, W/A/KV=4.5/4.5/4.52026.0457.59
SERQ-MXFP4Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.372026.0357.5
SpinQuantBackbone=LLaMA-3.2 3B, Calib. Time=∼156m2026.0356.88
DenseModel Size=Medium, Model=Llama-3.2-1B, Compression Factor (CF)=2x, Evaluation Protocol=Zero-shot2025.1256.54
FP16Family=Llama 3.2, Model=Llama-3.2-1B, W/A/KV=16/16/162026.0456.48
BLASTModel Size=Large, Model=Llama-7B, Compression Factor (CF)=2x, Evaluation Protocol=Zero-shot2025.1256.23
QuaRotFamily=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=4/4/42026.0456.13
LLM.int4()Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=-†2026.0356.06
QuaRotBackbone=LLaMA-3.2 3B, Calib. Time=∼8m2026.0355.76
ResQFamily=Qwen2.5, Model=Qwen2.5-1.5B, W/A/KV=4.5/4.5/4.52026.0455.66
QUIKFamily=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=4.5/4.5/4.52026.0455.56
L2QERBackbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=11.442026.0355.44
baselineBackbone=LLaMA-3.2 1B, #Bits=FP16, #Eff. (w)=162026.0354.82
L2QERBackbone=Qwen-2.5-3B, #Bits=W4A8, #Eff. (w)=4.352026.0354.71
FP16Family=Qwen2.5, Model=Qwen2.5-0.5B, W/A/KV=16/16/162026.0454.02
SERQ (GPTQ)Backbone=LLaMA-3.2 1B, #Bits=W4A8, #Eff. (w)=4.242026.0353.93
SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.0653.44
SERQ (RTN)Backbone=LLaMA-3.2 1B, #Bits=W4A8, #Eff. (w)=4.242026.0352.76