List your company
ResearchFormerly WizWand

Zero-shot Evaluation on 6 Downstream Tasks

80.05Average AccuracyDense
33.426845.530957.63569.7391Feb 6, 2026Feb 22, 2026Mar 11, 2026Mar 28, 2026Apr 14, 2026May 1, 2026May 18, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-3, Model Size=70B2026.0280.05
DenseWeight Update=None, Sparsity=0, Model Family=Qwen2.5, Model Size=72B2026.0278.69
DenseWeight Update=None, Sparsity=0, Model Family=Qwen2.5, Model Size=14B2026.0275.81
DenseWeight Update=None, Sparsity=0, Model Family=Qwen2.5, Model Size=32B2026.0275.12
Gemma2-9B (Baseline)MP=false, bits=16, Backbone=Gemma2-9B2026.0274.89
FPBackbone=Llama3-8b, Weight Bits=Full precision, Activation Bits=Full precision2026.0574.3
FPBackbone=Llama2-13b, Weight Bits=Full precision, Activation Bits=Full precision2026.0573.3
ScaleBITS + RTNMP=true, bits=3.1, Backbone=Gemma2-9B2026.0272.86
DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-3, Model Size=8B2026.0272.72
DenseWeight Update=None, Sparsity=0, Model Family=Qwen2.5, Model Size=7B2026.0272.18
Llama3.1-8B-Instruct BaselineMP=false, bits=16, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.0271.85
RTN-g128MP=false, bits=3.1, Backbone=Gemma2-9B2026.0271.4
GPTAQBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.0571.34
GPTQBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.0571.3
GPTAQ-MARRBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.0571.19
ResCompBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.0571.09
GPTQBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.0570.62
ResComp-MARRBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.0570.57
FPBackbone=Llama2-7b, Weight Bits=Full precision, Activation Bits=Full precision2026.0570.5
GPTAQBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.0570.3
ResComp-MARRBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.0570.19
ResCompBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.0570.11
GPTAQ-MARRBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.0569.3
ScaleBITS + RTNMP=true, bits=3.1, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.0269.23
GPTAQ-MARRBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.0567.83
GPTQBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.0567.67
ResCompBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.0567.67
GPTAQBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.0567.44
ResComp-MARRBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.0567.25
DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-2, Model Size=70B2026.0267.08
ScaleBITS + RTNMP=true, bits=2.1, Backbone=Gemma2-9B2026.0267.05
OA-EMModel=Llama 8B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0465.6
GreedyModel=Llama 8B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0464.9
RTN-g128MP=false, bits=3.1, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.0263.7
DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-2, Model Size=13B2026.0263.03
ScaleBITS + RTNMP=true, bits=2.1, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.0261.79
GreedyModel=Qwen 3B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0460.6
OA-EMModel=Qwen 3B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0460.3
DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-2, Model Size=7B2026.0259.71
OA-EMModel=Llama 3B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0459.1
OA-EMModel=Llama 3B, Beam width (b)=16, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0459.1
OA-EMModel=Llama 3B, Beam width (b)=4, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0458.9
GreedyModel=Llama 3B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0458.5
GreedyModel=Llama 3B, Beam width (b)=16, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0458.5
GreedyModel=Llama 3B, Beam width (b)=4, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0457.3
OA-EMModel=Llama 3B, Beam width (b)=8, Early stopping epochs (e)=5, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0457.2
GPTAQ-MARRBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.0556.59
GreedyModel=Llama 3B, Beam width (b)=8, Early stopping epochs (e)=5, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.0456.3
ResCompBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.0556.09
GPTAQBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.0554.51
ResComp-MARRBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.0554.23
GPTQBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.0552.99
GPTAQ-MARRBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.0552.52
ResComp-MARRBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.0552.13
GPTAQBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.0552.03
GPTAQBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.0551.92
ResCompBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.0551.85
ResCompBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.0550.67
GPTAQ-MARRBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.0550.09
ResComp-MARRBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.0548.63
GPTQBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.0548.52
RTNBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.0547.72
GPTQBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.0545.1
RTNBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.0543.64
RTN-g128MP=false, bits=2.1, Backbone=Gemma2-9B2026.0239.31
RTN-g128MP=false, bits=2.1, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.0237.45
RTNBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.0537.32
RTNBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.0536.3
RTNBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.0536.11
RTNBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.0535.22