List your company
ResearchFormerly WizWand

Zero-shot Evaluation on Evaluation Tasks Zero-shot Aggregate

79.95Avg. AccuracyBaseline
28.761242.050655.3468.6294Feb 5, 2026Feb 22, 2026Mar 11, 2026Mar 28, 2026Apr 14, 2026May 1, 2026May 19, 2026
Updated 5mo ago

Evaluation results

MethodLinks
BaselineBackbone=LLaMA3-70B, Precision=FP162026.0579.95
TORQBackbone=LLaMA3-70B2026.0576.28
FP16Model=Phi3-3.8B, Quantization Bits=None2026.0275.41
BaselineBackbone=Qwen3-32B, Precision=FP162026.0574.82
FP16Model=Qwen3-8B, Quantization Bits=None2026.0274.15
OSTQuantBackbone=LLaMA3-70B2026.0573.93
BaselineBackbone=Qwen3-14B, Precision=FP162026.0573.81
TORQBackbone=Qwen3-32B2026.0573.63
GlowQ-SZero-shot=true, Model=Vicuna-13B2026.0373.24
BaselineBackbone=LLaMA3-8B, Precision=FP162026.0573.18
QERAZero-shot=true, Model=Vicuna-13B2026.0371.9
GlowQZero-shot=true, Model=Vicuna-13B2026.0371.86
FP16Rank=-, Zero-shot=true, Model=Vicuna-13B2026.0371.76
ZeroQuant-V2Zero-shot=true, Model=Vicuna-13B2026.0371.67
L2QERRank=64, Zero-shot=true, Model=Vicuna-13B2026.0371.67
SNRQ-SModel=L2-13B, Quantization Time (s)=786.42026.0271.11
SNRQ-CModel=L2-13B, Quantization Time (s)=965.02026.0271.06
GPTAQModel=L2-13B, Quantization Time (s)=1012.02026.0271
SNRQ-S+RHTModel=L2-13B, Quantization Time (s)=970.22026.0270.9
GPTAQ+RHTModel=L2-13B, Quantization Time (s)=1259.42026.0270.86
QuIPModel=L2-13B, Quantization Time (s)=1096.22026.0270.76
SNRQ-C+RHTModel=L2-13B, Quantization Time (s)=1106.32026.0270.71
SNRQ-SModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=478.42026.0270.56
TORQBackbone=Qwen3-14B2026.0570.51
BaselineBackbone=Qwen3-8B, Precision=FP162026.0570.43
SNRQ-CModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=480.22026.0270.05
FlatQuantBackbone=Qwen3-14B2026.0569.87
SNRQ-S+RHTModel=L3-8B, Quantization Time (s)=688.72026.0269.78
SNRQ-SModel=L3-8B, Quantization Time (s)=542.82026.0269.33
TORQBackbone=LLaMA3-8B2026.0569.32
LDLQModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=419.52026.0269.26
SNRQ-C+RHTModel=L3-8B, Quantization Time (s)=741.82026.0269.01
SNRQ-SModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=267.72026.0269
GPTAQModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=670.42026.0268.94
GPTAQModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=3322026.0268.74
GPTAQ+RHTModel=L3-8B, Quantization Time (s)=853.12026.0268.73
GPTQModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=494.92026.0268.49
SNRQ-CModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=271.42026.0268.47
FlatQuantBackbone=LLaMA3-8B2026.0568.34
GPTQModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=262.72026.0267.84
BRQBackbone=LLaMA3-8B2026.0567.76
TORQBackbone=Qwen3-8B2026.0567.75
QuIPModel=L3-8B, Quantization Time (s)=763.82026.0267.6
SNRQ-CModel=L3-8B, Quantization Time (s)=594.42026.0267.6
SNRQ-SModel=L2-7B, Quantization Time (s)=449.02026.0267.34
SNRQ-C+RHTModel=L2-7B, Quantization Time (s)=697.32026.0267.26
OSTQuantBackbone=Qwen3-32B2026.0567.19
SNRQ-S+RHTModel=L2-7B, Quantization Time (s)=644.52026.0267.18
GPTAQ+RHTModel=L2-7B, Quantization Time (s)=795.72026.0267.02
LDLQModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=231.92026.0267.01
FlatQuantBackbone=Qwen3-8B2026.0566.96
SNRQ-CModel=L2-7B, Quantization Time (s)=517.12026.0266.82
GPTAQModel=L2-7B, Quantization Time (s)=590.42026.0266.34
QuIPModel=L2-7B, Quantization Time (s)=704.82026.0266.22
OSTQuantBackbone=LLaMA3-8B2026.0565.92
GPTAQModel=L3-8B, Quantization Time (s)=715.02026.0265.77
BRQBackbone=Qwen3-8B2026.0564.73
OSTQuantBackbone=Qwen3-14B2026.0564.06
QuaRotBackbone=LLaMA3-70B2026.0562.87
QuaRotBackbone=LLaMA3-8B2026.0562.13
QuaRotBackbone=Qwen3-32B2026.0561.23
QuaRotBackbone=Qwen3-14B2026.0560.21
GPTQBackbone=Qwen3-32B2026.0557.62
QuaRotBackbone=Qwen3-8B2026.0555.35
OSTQuantBackbone=Qwen3-8B2026.0554.4
GPTQBackbone=Qwen3-14B2026.0539.91
RTNBackbone=Qwen3-32B2026.0538.4
GPTQBackbone=Qwen3-8B2026.0535.54
RTNBackbone=Qwen3-14B2026.0535.22
GPTQBackbone=LLaMA3-70B2026.0534.82
GPTQBackbone=LLaMA3-8B2026.0533.6
RTNBackbone=LLaMA3-8B2026.0532.03
RTNBackbone=LLaMA3-70B2026.0530.92
RTNBackbone=Qwen3-8B2026.0530.73