Question Answering on Eight QA datasets Average
65.2AccuracyFP Model
Evaluation Results
| Method | Links | |
|---|---|---|
| FP ModelModel Backbone=LLaMA-3B, Weight Precision (bits)=Full, Activation Precision (bits)=Full2026.05 | 65.2 | |
| ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.05 | 58.7 | |
| WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.05 | 58.6 | |
| WINQModel Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 58.6 | |
| FP ModelModel Backbone=LLaMA-1B, Weight Precision (bits)=Full, Activation Precision (bits)=Full2026.05 | 58.5 | |
| WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.05 | 57.8 | |
| ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.05 | 57.2 | |
| ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.05 | 56.7 | |
| WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.05 | 56.6 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.05 | 56.5 | |
| AWQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.05 | 56.4 | |
| ParetoQModel Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 55.9 | |
| WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.05 | 55.6 | |
| WINQModel Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 55.2 | |
| ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.05 | 54.7 | |
| ParetoQModel Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 54 | |
| WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.05 | 53 | |
| GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.05 | 52.8 | |
| WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.05 | 52.6 | |
| WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 52.5 | |
| RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=4, Activation Precision (bits)=162026.05 | 52.4 | |
| ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.05 | 52.2 | |
| ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.05 | 51.9 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.05 | 51.9 | |
| ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 51.9 | |
| WINQModel Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 49 | |
| ParetoQModel Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 48.2 | |
| AWQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.05 | 42 | |
| GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.05 | 41.1 | |
| RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=3, Activation Precision (bits)=162026.05 | 38.8 | |
| RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.05 | 38.5 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.05 | 38.3 | |
| GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.05 | 36.8 | |
| AWQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=162026.05 | 36.4 | |
| RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.05 | 36.2 | |
| RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 36.2 | |
| RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.05 | 36.1 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.05 | 34.9 | |
| RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.05 | 33.7 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.05 | 33.7 | |
| RTN (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 33.7 | |
| RTN (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 33.7 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 33.3 | |
| RTN (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 33.2 | |
| GPTQ (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 33.1 | |
| GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.05 | 32.8 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 32.8 | |
| GPTQ (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 32.8 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-3B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 32.8 | |
| GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=162026.05 | 32.7 | |
| GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 32.7 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=82026.05 | 32.7 | |
| GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=2, Activation Precision (bits)=82026.05 | 32.7 | |
| SpinQuant (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1.58, Activation Precision (bits)=162026.05 | 32.6 | |
| GPTQ (PTQ)Model Backbone=LLaMA-1B, Weight Precision (bits)=1, Activation Precision (bits)=82026.05 | 31.7 |