List your company
ResearchFormerly WizWand

Zero-shot Task Evaluation on tasks 0-shot

68.58AccuracyFP16
33.16842.361551.55560.7485May 11, 2026May 14, 2026May 18, 2026May 22, 2026May 25, 2026May 29, 2026Jun 2, 2026
Updated 4mo ago

Evaluation results

MethodLinks
FP16Backbone=3-82026.0668.58
FP16Backbone=2-132026.0667.81
FP16Backbone=2-72026.0664.85
BF16 (dense)Model=LLaMA-3 8B2026.0563.94
ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0560.86
GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0560.55
LiftUQType=LQ-16/10, Bits=1.62, Backbone=2-132026.0660.54
ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0560.52
ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0560.26
ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0560.17
GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0559.72
GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0559.44
GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0559.02
RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0558.71
RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0558.55
ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0558.34
ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0557.91
ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0557.59
ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0557.29
GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0557.14
LiftUQType=LQ-16/10, Bits=1.62, Backbone=3-82026.0656.66
RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0556.61
GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0556.51
GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0556.31
RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0556.22
LiftUQType=LQ-16/10, Bits=1.62, Backbone=2-72026.0656.19
GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0556.08
BF16 (dense)Model=LLaMA-3.2 1B-Instruct2026.0553.22
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0549.72
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0549.38
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0549.36
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0549.31
PTQ1.61Type=UQ, Bits=1.61, Backbone=2-132026.0649.21
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0549.01
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0548.77
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0548.52
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0548.27
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0547.74
ADMM-QModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0547.55
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0547.26
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0547.18
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0546.9
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0546.84
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0546.75
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0546.7
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0546.15
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0545.88
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0545.78
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0545.76
ADMM-QModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0545.63
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0545.62
ADMM-QModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0544.79
ADMM-QModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0544.64
PTQ1.61Type=UQ, Bits=1.61, Backbone=2-72026.0644.14
PTQ1.61Type=UQ, Bits=1.61, Backbone=3-82026.0643.99
GPTQModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0542.7
GPTQModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0542.44
RTNModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0540.85
GPTQModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0540.47
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0540.05
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0539.86
GPTQModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0539.52
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0539.26
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0539.09
RTNModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0538.81
ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0538.67
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0538.61
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0538.2
RTNModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0538.01
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0537.74
RTNModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0537.69
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0537.54
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0537.14
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0536.99
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0536.94
GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0536.85
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0536.67
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0536.05
RTNModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.0535.48
RTNModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.0535.47
RTNModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0534.73
RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.0534.58
RTNModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.0534.53