Language Modeling on C4 (Perplexity)
1PerplexityWanda
Evaluation Results
| Method | Links | |
|---|---|---|
| WandaWall-clock (h)=0.159, #GPUs=1, Model=LLaMA-2-7B, Sparsity=90%2025.10 | 1 | |
| W16A16 BaselineBits=W16A16, Model=Llama2-70B2026.03 | 5.54 | |
| SliderQuantBits=W4A16, Model=Llama2-70B2026.03 | 5.6 | |
| CBQBits=W4A16, Model=Llama2-70B2026.03 | 5.64 | |
| OmniQuantBits=W4A16, Model=Llama2-70B2026.03 | 5.65 | |
| GPTQBits=W4A16, Model=Llama2-70B2026.03 | 5.67 | |
| AWQBits=W4A16, Model=Llama2-70B2026.03 | 5.7 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 5.72 | |
| RTNBits=W4A16, Model=Llama2-70B2026.03 | 5.79 | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 5.88 | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 5.92 | |
| SpinQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 6.07 | |
| QuaRot#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 6.12 | |
| FPBackbone=Llama2-13b, Weight Bits=Full precision, Activation Bits=Full precision2026.05 | 6.41 | |
| FPModel=Llama2-13b, W/A=FP2026.05 | 6.41 | |
| W16A16 BaselineBits=W16A16, Model=Llama2-13B2026.03 | 6.46 | |
| BaselinePrecision=FP16, Model=LLaMA2-13B2026.05 | 6.46 | |
| BaselinePrec.=FP16, Model Scale=LLaMA2-13B2026.05 | 6.46 | |
| FP16Model=Llama-2-13B2026.03 | 6.47 | |
| BF16Backbone=Llama-2-13B, Quantization=None2026.05 | 6.47 | |
| SliderQuantBits=W4A16, Model=Llama2-13B2026.03 | 6.54 | |
| OmniQuantBits=W4A16, Model=Llama2-13B2026.03 | 6.65 | |
| CBQBits=W4A16, Model=Llama2-13B2026.03 | 6.67 | |
| GPTQBits=W4A16, Model=Llama2-13B2026.03 | 6.7 | |
| DenseRatio=1.0, zero-shot evaluation=true2026.04 | 6.73 | |
| FP16Model=L2-13B, Calibration samples=1282026.04 | 6.73 | |
| FP16Model=L2-13B, Bits=16-bit2026.04 | 6.73 | |
| FP16Precision=FP16, Model Architecture=L2-13B2026.04 | 6.73 | |
| FP16Model=L2-13B2026.04 | 6.73 | |
| LLaMA2-13BBackbone=LLaMA2-13B, Bit=-, T × D=-, R=-, FLOPs(T)=13.42, Power(J)=61.742026.04 | 6.73 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 6.73 | |
| AWQBits=W4A16, Model=Llama2-13B2026.03 | 6.74 | |
| OSAQ+WKVQuantPrecision=W4A16KV4, Model=LLaMA2-13B2026.05 | 6.81 | |
| RTNBits=W4A16, Model=Llama2-13B2026.03 | 6.83 | |
| OSAQ+QuaRotPrec.=W4A4, Model Scale=LLaMA2-13B2026.05 | 6.85 | |
| OSAQ+SpinQuantPrec.=W4A4, Model Scale=LLaMA2-13B2026.05 | 6.86 | |
| WKVQuantPrecision=W4A16KV4, Model=LLaMA2-13B2026.05 | 6.89 | |
| FPBackbone=Llama2-7b, Weight Bits=Full precision, Activation Bits=Full precision2026.05 | 6.9 | |
| FPModel=Llama2-7b, W/A=FP2026.05 | 6.9 | |
| SpinQuantPrec.=W4A4, Model Scale=LLaMA2-13B2026.05 | 6.93 | |
| OSAQ+DuQuantPrec.=W4A4, Model Scale=LLaMA2-13B2026.05 | 6.96 | |
| W16A16 BaselineBits=W16A16, Model=Llama2-7B2026.03 | 6.97 | |
| FP16Model Architecture=Llama-2-7B2026.04 | 6.97 | |
| LLAMA-2-7BModel=LLAMA-2-7B, Bits=FP162026.04 | 6.97 | |
| BaselinePrecision=FP16, Model=LLaMA2-7B2026.05 | 6.97 | |
| BaselinePrec.=FP16, Model Scale=LLaMA2-7B2026.05 | 6.97 | |
| QuaRotPrec.=W4A4, Model Scale=LLaMA2-13B2026.05 | 6.97 | |
| BF16Backbone=LLaMA-2-7B, Quantization=None2026.05 | 6.97 | |
| DuQuantPrec.=W4A4, Model Scale=LLaMA2-13B2026.05 | 7.05 | |
| FP16Bits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 7.06 | |
| FP16Bits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 7.06 | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 7.07 | |
| LLAMA-1-7BModel=LLAMA-1-7B, Bits=FP162026.04 | 7.08 | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 7.11 | |
| SDLLMBackbone=LLaMA2-13B, Bit=4-1.58, T × D=1 × 16, R=0.215, FLOPs(T)=2.89, Power(J)=2.602026.04 | 7.15 | |
| FP16Model=L3-70B, Bits=16-bit2026.04 | 7.17 | |
| FP16Precision=FP16, Model Architecture=L3-70B2026.04 | 7.17 | |
| FP16Model=L3-70B2026.04 | 7.17 | |
| GPTAQ-MARRModel=Llama2-13b, W/A=W4A42026.05 | 7.17 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 7.17 | |
| FP16#Bits W-A-KV=3-3-3, Model=LLaMA3-70B2026.05 | 7.17 | |
| FP16#Bits W-A-KV=2-4-16, Model=LLaMA3-70B2026.05 | 7.17 | |
| FP16Bits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 7.17 | |
| FP16Bits (W-A-KV)=2-4-16, Model=LLaMA3-70B, Zero-shot=true2026.05 | 7.17 | |
| SliderQuantBits=W4A16, Model=Llama2-7B2026.03 | 7.19 | |
| GPTAQModel=Llama2-13b, W/A=W4A42026.05 | 7.22 | |
| CBQBits=W4A16, Model=Llama2-7B2026.03 | 7.23 | |
| DenseSparsity=0%, Backbone=LLaMA-2-7B2025.10 | 7.26 | |
| FP16Model=L2-7B, Calibration samples=1282026.04 | 7.26 | |
| FP16Model=L3-8B, Calibration samples=1282026.04 | 7.26 | |
| FP16Model=L2-7B, Bits=16-bit2026.04 | 7.26 | |
| FP16Precision=FP16, Model Architecture=L2-7B2026.04 | 7.26 | |
| FP16Model=L2-7B2026.04 | 7.26 | |
| LLaMA2-7BBackbone=LLaMA2-7B, Bit=-, T × D=-, R=-, FLOPs(T)=6.91, Power(J)=31.772026.04 | 7.26 | |
| ResComp-MARRModel=Llama2-13b, W/A=W4A42026.05 | 7.26 | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA2-7B2026.05 | 7.26 | |
| OriginalParam Ratio=1.0, Backbone=LLaMA-2-7B2026.04 | 7.2745 | |
| ResCompModel=Llama2-13b, W/A=W4A42026.05 | 7.28 | |
| OmniQuantPrecision=W4A16KV4, Model=LLaMA2-13B2026.05 | 7.3 | |
| BaselineRatio=1.0, Memory Footprint (GB)=12.6GB2026.04 | 7.34 | |
| DenseCompression Ratio=1.0, Backbone=LLaMA-7B2026.04 | 7.34 | |
| GPTAQModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 7.34 | |
| GPTAQ+OursModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 7.34 | |
| BaselineCompression Ratio=0.02026.05 | 7.34 | |
| OmniQuantBits=W4A16, Model=Llama2-7B2026.03 | 7.35 | |
| GPTQBits=W4A16, Model=Llama2-7B2026.03 | 7.37 | |
| GPTQ+OursModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 7.37 | |
| GPTQModel=Llama2-13b, W/A=W4A42026.05 | 7.37 | |
| GPTQModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 7.38 | |
| BF16Models=Mixtral-8x7B, Quantization Strategy=Embedding-wise, Quantization Bit-width=BF162026.04 | 7.41 | |
| OursBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 7.41 | |
| OSAQ+WKVQuantPrecision=W4A16KV4, Model=LLaMA2-7B2026.05 | 7.43 | |
| GPTAQ-MARRBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 7.45 | |
| SpinQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 7.48 | |
| WKVQuantPrecision=W4A16KV4, Model=LLaMA2-7B2026.05 | 7.49 | |
| GPTAQBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 7.5 | |
| ResComp-MARRBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 7.54 | |
| QuaRot#Bits W-A-KV=4-4-4, Model=LLaMA2-13B2026.05 | 7.54 | |
| AWQModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 7.57 | |
| ResCompBackbone=Llama2-13b, Weight Bits=3, Activation Bits=162026.05 | 7.58 |