Zero-shot Evaluation on 6 Downstream Tasks
80.05Average AccuracyDense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-3, Model Size=70B2026.02 | 80.05 | |
| DenseWeight Update=None, Sparsity=0, Model Family=Qwen2.5, Model Size=72B2026.02 | 78.69 | |
| DenseWeight Update=None, Sparsity=0, Model Family=Qwen2.5, Model Size=14B2026.02 | 75.81 | |
| DenseWeight Update=None, Sparsity=0, Model Family=Qwen2.5, Model Size=32B2026.02 | 75.12 | |
| Gemma2-9B (Baseline)MP=false, bits=16, Backbone=Gemma2-9B2026.02 | 74.89 | |
| FPBackbone=Llama3-8b, Weight Bits=Full precision, Activation Bits=Full precision2026.05 | 74.3 | |
| FPBackbone=Llama2-13b, Weight Bits=Full precision, Activation Bits=Full precision2026.05 | 73.3 | |
| ScaleBITS + RTNMP=true, bits=3.1, Backbone=Gemma2-9B2026.02 | 72.86 | |
| DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-3, Model Size=8B2026.02 | 72.72 | |
| DenseWeight Update=None, Sparsity=0, Model Family=Qwen2.5, Model Size=7B2026.02 | 72.18 | |
| Llama3.1-8B-Instruct BaselineMP=false, bits=16, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.02 | 71.85 | |
| RTN-g128MP=false, bits=3.1, Backbone=Gemma2-9B2026.02 | 71.4 | |
| GPTAQBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 71.34 | |
| GPTQBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 71.3 | |
| GPTAQ-MARRBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 71.19 | |
| ResCompBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 71.09 | |
| GPTQBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.05 | 70.62 | |
| ResComp-MARRBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 70.57 | |
| FPBackbone=Llama2-7b, Weight Bits=Full precision, Activation Bits=Full precision2026.05 | 70.5 | |
| GPTAQBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.05 | 70.3 | |
| ResComp-MARRBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.05 | 70.19 | |
| ResCompBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.05 | 70.11 | |
| GPTAQ-MARRBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.05 | 69.3 | |
| ScaleBITS + RTNMP=true, bits=3.1, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.02 | 69.23 | |
| GPTAQ-MARRBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.05 | 67.83 | |
| GPTQBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.05 | 67.67 | |
| ResCompBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.05 | 67.67 | |
| GPTAQBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.05 | 67.44 | |
| ResComp-MARRBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.05 | 67.25 | |
| DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-2, Model Size=70B2026.02 | 67.08 | |
| ScaleBITS + RTNMP=true, bits=2.1, Backbone=Gemma2-9B2026.02 | 67.05 | |
| OA-EMModel=Llama 8B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 65.6 | |
| GreedyModel=Llama 8B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 64.9 | |
| RTN-g128MP=false, bits=3.1, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.02 | 63.7 | |
| DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-2, Model Size=13B2026.02 | 63.03 | |
| ScaleBITS + RTNMP=true, bits=2.1, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.02 | 61.79 | |
| GreedyModel=Qwen 3B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 60.6 | |
| OA-EMModel=Qwen 3B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 60.3 | |
| DenseWeight Update=None, Sparsity=0, Model Family=LLaMA-2, Model Size=7B2026.02 | 59.71 | |
| OA-EMModel=Llama 3B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 59.1 | |
| OA-EMModel=Llama 3B, Beam width (b)=16, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 59.1 | |
| OA-EMModel=Llama 3B, Beam width (b)=4, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 58.9 | |
| GreedyModel=Llama 3B, Beam width (b)=8, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 58.5 | |
| GreedyModel=Llama 3B, Beam width (b)=16, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 58.5 | |
| GreedyModel=Llama 3B, Beam width (b)=4, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 57.3 | |
| OA-EMModel=Llama 3B, Beam width (b)=8, Early stopping epochs (e)=5, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 57.2 | |
| GPTAQ-MARRBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.05 | 56.59 | |
| GreedyModel=Llama 3B, Beam width (b)=8, Early stopping epochs (e)=5, Bitrate=2 bpp, Protocol=Post-PV-tuning2026.04 | 56.3 | |
| ResCompBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.05 | 56.09 | |
| GPTAQBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.05 | 54.51 | |
| ResComp-MARRBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.05 | 54.23 | |
| GPTQBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.05 | 52.99 | |
| GPTAQ-MARRBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.05 | 52.52 | |
| ResComp-MARRBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.05 | 52.13 | |
| GPTAQBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.05 | 52.03 | |
| GPTAQBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.05 | 51.92 | |
| ResCompBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.05 | 51.85 | |
| ResCompBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.05 | 50.67 | |
| GPTAQ-MARRBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.05 | 50.09 | |
| ResComp-MARRBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.05 | 48.63 | |
| GPTQBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.05 | 48.52 | |
| RTNBackbone=Llama3-8b, Weight Bits=3, Activation Bits=162026.05 | 47.72 | |
| GPTQBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.05 | 45.1 | |
| RTNBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 43.64 | |
| RTN-g128MP=false, bits=2.1, Backbone=Gemma2-9B2026.02 | 39.31 | |
| RTN-g128MP=false, bits=2.1, Backbone=Llama3.1-8B-Instruct, Zero-shot protocol=Average over six tasks2026.02 | 37.45 | |
| RTNBackbone=Llama2-7b, Weight Bits=3, Activation Bits=162026.05 | 37.32 | |
| RTNBackbone=Llama2-7b, Weight Bits=2, Activation Bits=162026.05 | 36.3 | |
| RTNBackbone=Llama3-8b, Weight Bits=2, Activation Bits=162026.05 | 36.11 | |
| RTNBackbone=Llama2-13b, Weight Bits=2, Activation Bits=162026.05 | 35.22 |