Language Generation on C4
5.52PerplexityBaseline
Evaluation results
| Method | Links | |
|---|---|---|
| BaselinePrec.=FP16, Model=2-70B2026.05 | 5.52 | |
| OSAQ+GPTQPrec.=W4A16, Model=2-70B2026.05 | 5.61 | |
| -#Bits=W16A16, Model=Llama-65B2026.03 | 5.62 | |
| OmniQuantPrec.=W4A16, Model=2-70B2026.05 | 5.65 | |
| OSAQ+AWQPrec.=W4A16, Model=2-70B2026.05 | 5.66 | |
| GPTQPrec.=W4A16, Model=2-70B2026.05 | 5.67 | |
| SliderQuant#Bits=W4A16, Model=Llama-65B2026.03 | 5.7 | |
| AWQPrec.=W4A16, Model=2-70B2026.05 | 5.71 | |
| CBQ#Bits=W4A16, Model=Llama-65B2026.03 | 5.72 | |
| MagRPrec.=W4A16, Model=2-70B2026.05 | 5.72 | |
| OSAQ+AWQPrec.=W3A16 g128, Model=2-70B2026.05 | 5.72 | |
| OmniQuant#Bits=W4A16, Model=Llama-65B2026.03 | 5.73 | |
| AWQ#Bits=W4A16, Model=Llama-65B2026.03 | 5.77 | |
| OSAQ+GPTQPrec.=W3A16 g128, Model=2-70B2026.05 | 5.78 | |
| RTNPrec.=W4A16, Model=2-70B2026.05 | 5.79 | |
| GPTQ#Bits=W4A16, Model=Llama-65B2026.03 | 5.8 | |
| AWQPrec.=W3A16 g128, Model=2-70B2026.05 | 5.82 | |
| RTN#Bits=W4A16, Model=Llama-65B2026.03 | 5.85 | |
| OmniQuantPrec.=W3A16 g128, Model=2-70B2026.05 | 5.85 | |
| GPTQPrec.=W3A16 g128, Model=2-70B2026.05 | 5.85 | |
| QuIPPrec.=W4A16, Model=2-70B2026.05 | 5.87 | |
| MagRPrec.=W3A16 g128, Model=2-70B2026.05 | 6 | |
| RTNPrec.=W3A16 g128, Model=2-70B2026.05 | 6.02 | |
| OmniQuantPrec.=W3A16, Model=2-70B2026.05 | 6.06 | |
| OSAQ+QuIPPrec.=W3A16, Model=2-70B2026.05 | 6.06 | |
| OSAQ+GPTQPrec.=W3A16, Model=2-70B2026.05 | 6.09 | |
| QuIPPrec.=W3A16, Model=2-70B2026.05 | 6.14 | |
| BaselinePrec.=FP16, Model=2-13B2026.05 | 6.46 | |
| FP16Avg. Bit=16, Backbone=LLaMA2-13B2024.10 | 6.47 | |
| CMPQAvg. Bit=4, Backbone=LLaMA2-13B2024.10 | 6.55 | |
| AWQAvg. Bit=4, Backbone=LLaMA2-13B2024.10 | 6.56 | |
| GPTQPrec.=W3A16, Model=2-70B2026.05 | 6.57 | |
| LLM-MQAvg. Bit=4, Backbone=LLaMA2-13B2024.10 | 6.58 | |
| QuIP#Avg. Bit=4, Backbone=LLaMA2-13B2024.10 | 6.59 | |
| VPTQAvg. Bit=4, Backbone=LLaMA2-13B2024.10 | 6.59 | |
| -#Bits=W16A16, Model=Llama-13B2026.03 | 6.61 | |
| OSAQ+GPTQPrec.=W4A16, Model=2-13B2026.05 | 6.64 | |
| OmniQuantPrec.=W4A16, Model=2-13B2026.05 | 6.65 | |
| OSAQ+AWQPrec.=W4A16, Model=2-13B2026.05 | 6.67 | |
| GPTQPrec.=W4A16, Model=2-13B2026.05 | 6.7 | |
| SliderQuant#Bits=W4A16, Model=Llama-13B2026.03 | 6.73 | |
| AWQPrec.=W4A16, Model=2-13B2026.05 | 6.74 | |
| GPTQAvg. Bit=4, Backbone=LLaMA2-13B2024.10 | 6.74 | |
| OmniQuant#Bits=W4A16, Model=Llama-13B2026.03 | 6.76 | |
| CBQ#Bits=W4A16, Model=Llama-13B2026.03 | 6.77 | |
| MagRPrec.=W3A16, Model=2-70B2026.05 | 6.77 | |
| MagRPrec.=W4A16, Model=2-13B2026.05 | 6.81 | |
| RTNPrec.=W4A16, Model=2-13B2026.05 | 6.83 | |
| RTNAvg. Bit=4, Backbone=LLaMA2-13B2024.10 | 6.83 | |
| QuIPAvg. Bit=4, Backbone=LLaMA2-13B2024.10 | 6.83 | |
| GPTQ#Bits=W4A16, Model=Llama-13B2026.03 | 6.84 | |
| QuIPPrec.=W4A16, Model=2-13B2026.05 | 6.88 | |
| OSAQ+AWQPrec.=W3A16 g128, Model=2-13B2026.05 | 6.89 | |
| BaselinePrec.=FP16, Model=3-70B2026.05 | 6.9 | |
| CMPQAvg. Bit=3, Backbone=LLaMA2-13B2024.10 | 6.93 | |
| AWQPrec.=W3A16 g128, Model=2-13B2026.05 | 6.94 | |
| AWQAvg. Bit=3, Backbone=LLaMA2-13B2024.10 | 6.94 | |
| OSAQ+GPTQPrec.=W3A16 g128, Model=2-13B2026.05 | 6.95 | |
| BaselinePrec.=FP16, Model=2-7B2026.05 | 6.97 | |
| Full-precision#Bits=16/16, Size (GB)=13.542026.05 | 6.97 | |
| FP16Avg. Bit=16, Backbone=LLaMA2-7B2024.10 | 6.97 | |
| RTN#Bits=W4A16, Model=Llama-13B2026.03 | 6.98 | |
| AWQ#Bits=W4A16, Model=Llama-13B2026.03 | 6.98 | |
| OmniQuantPrec.=W3A16 g128, Model=2-13B2026.05 | 6.98 | |
| QuIP#Avg. Bit=3, Backbone=LLaMA2-13B2024.10 | 6.98 | |
| GPTQPrec.=W3A16 g128, Model=2-13B2026.05 | 7 | |
| VPTQAvg. Bit=3, Backbone=LLaMA2-13B2024.10 | 7.03 | |
| GPTVQAvg. Bit=3, Backbone=LLaMA2-13B2024.10 | 7.05 | |
| GPTQAvg. Bit=3, Backbone=LLaMA2-13B2024.10 | 7.06 | |
| -#Bits=W16A16, Model=Llama-7B2026.03 | 7.08 | |
| QuIPPrec.=W4A16, Model=3-70B2026.05 | 7.1 | |
| CMPQAvg. Bit=4, Backbone=LLaMA2-7B2024.10 | 7.1 | |
| AWQPrec.=W3A16, Model=2-70B2026.05 | 7.11 | |
| MagRPrec.=W3A16 g128, Model=2-13B2026.05 | 7.12 | |
| QuIPAvg. Bit=4, Backbone=LLaMA2-7B2024.10 | 7.12 | |
| AWQAvg. Bit=4, Backbone=LLaMA2-7B2024.10 | 7.13 | |
| VPTQAvg. Bit=4, Backbone=LLaMA2-7B2024.10 | 7.13 | |
| QuIP#Avg. Bit=4, Backbone=LLaMA2-7B2024.10 | 7.17 | |
| RTNPrec.=W3A16 g128, Model=2-13B2026.05 | 7.18 | |
| OSAQ+AWQPrec.=W4A16, Model=3-70B2026.05 | 7.22 | |
| LLM-MQAvg. Bit=4, Backbone=LLaMA2-7B2024.10 | 7.22 | |
| GPTQAvg. Bit=4, Backbone=LLaMA2-7B2024.10 | 7.23 | |
| OSAQ+GPTQPrec.=W4A16, Model=3-70B2026.05 | 7.24 | |
| QuIPAvg. Bit=3, Backbone=LLaMA2-13B2024.10 | 7.25 | |
| GPTVQAvg. Bit=4, Backbone=LLaMA2-7B2024.10 | 7.25 | |
| SliderQuant#Bits=W4A16, Model=Llama-7B2026.03 | 7.26 | |
| GPTVQAvg. Bit=4, Backbone=LLaMA2-13B2024.10 | 7.26 | |
| OSAQ+QuIPPrec.=W3A16, Model=2-13B2026.05 | 7.27 | |
| CBQ#Bits=W4A16, Model=Llama-7B2026.03 | 7.33 | |
| OmniQuant#Bits=W4A16, Model=Llama-7B2026.03 | 7.34 | |
| OSAQ+GPTQPrec.=W4A16, Model=2-7B2026.05 | 7.34 | |
| QuIPPrec.=W3A16, Model=2-13B2026.05 | 7.34 | |
| OmniQuantPrec.=W4A16, Model=2-7B2026.05 | 7.35 | |
| GPTQPrec.=W4A16, Model=2-7B2026.05 | 7.37 | |
| AWQPrec.=W4A16, Model=3-70B2026.05 | 7.4 | |
| GPTQPrec.=W4A16, Model=3-70B2026.05 | 7.4 | |
| GPTQ#Bits=W4A16, Model=Llama-7B2026.03 | 7.43 | |
| OmniQuantPrec.=W3A16, Model=2-13B2026.05 | 7.44 | |
| OSAQ+AWQPrec.=W4A16, Model=2-7B2026.05 | 7.5 | |
| AWQ#Bits=W4A16, Model=Llama-7B2026.03 | 7.52 |