List your company
ResearchFormerly WizWand

Question Answering on Eight QA datasets Average

65.2AccuracyFP Model
30.3639.40548.4557.495May 17, 2026
Updated 5mo ago

Evaluation results

MethodLinks
FP ModelModel Backbone=LLaMA-3B, Weight Precision (bits)=Full, Activation Precision (bits)=Full2026.0565.2
ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.0558.7
WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.0558.6
WINQModel Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0558.6
FP ModelModel Backbone=LLaMA-1B, Weight Precision (bits)=Full, Activation Precision (bits)=Full2026.0558.5
WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.0557.8
ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.0557.2
ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.0556.7
WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.0556.6
SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.0556.5
AWQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.0556.4
ParetoQModel Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0555.9
WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.0555.6
WINQModel Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0555.2
ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.0554.7
ParetoQModel Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0554
WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.0553
GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.0552.8
WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.0552.6
WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0552.5
RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.0552.4
ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.0552.2
ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.0551.9
SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.0551.9
ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0551.9
WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0549
ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0548.2
AWQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.0542
GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.0541.1
RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.0538.8
RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.0538.5
SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.0538.3
GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.0536.8
AWQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.0536.4
RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.0536.2
RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0536.2
RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.0536.1
SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.0534.9
RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.0533.7
SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.0533.7
RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0533.7
RTN (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0533.7
SpinQuant (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0533.3
RTN (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0533.2
GPTQ (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0533.1
GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.0532.8
SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0532.8
GPTQ (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0532.8
SpinQuant (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0532.8
GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.0532.7
GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0532.7
SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.0532.7
GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.0532.7
SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.0532.6
GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.0531.7