Language Modeling on Wikitext2 (Perplexity)
2.58PerplexityDense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseSparsity=0%, Backbone=Llama-3 70B, Context Length=81922025.10 | 2.58 | |
| FP16Backbone=Llama-3-70B, Context Length=81922026.06 | 2.59 | |
| FP16Model=L3-70B, Bits=16-bit2026.04 | 2.85 | |
| DenseSparsity=0%, Backbone=Llama-2 70B, Context Length=40962025.10 | 3.12 | |
| FP16Backbone=Llama-2-70B, Context Length=20482026.06 | 3.32 | |
| LiftQuantBackbone=Llama-2-70B, Type=LQ-24/8, Bits=3.01-3.02, Context Length=20482026.06 | 3.35 | |
| LiftQuantBackbone=Llama-3-70B, Type=LQ-24/8, Bits=3.01-3.02, Context Length=81922026.06 | 3.45 | |
| VPTQ#Backbone=Llama-2-70B, Type=VQ, Bits=3.01-3.03, Context Length=20482026.06 | 3.55 | |
| QuIP#Backbone=Llama-2-70B, Type=VQ, Bits=3.00, Context Length=20482026.06 | 3.56 | |
| QuIP#Backbone=Llama-3-70B, Type=VQ, Bits=3.00, Context Length=81922026.06 | 3.59 | |
| EQAT-g128Backbone=Llama-2-70B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 3.61 | |
| QuarotBackbone=Llama-2-70B, Type=UQ, Bits=3.00, Context Length=20482026.06 | 3.72 | |
| LiftQuantBackbone=Llama-2-70B, Type=LQ-24/10, Bits=2.41-2.42, Context Length=20482026.06 | 3.78 | |
| EQAT-g128Backbone=Llama-3-70B, Type=UQ, Bits=3.13, Context Length=81922026.06 | 3.78 | |
| GPTQ-g128Backbone=Llama-2-70B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 3.85 | |
| QTIPBackbone=Llama-2-70B, Type=VQ, Bits=2.00, Context Length=20482026.06 | 3.94 | |
| LiftQuantBackbone=Llama-2-70B, Type=LQ-32/16, Bits=2.01-2.02, Context Length=20482026.06 | 4.08 | |
| LiftQuantBackbone=Llama-3-70B, Type=LQ-24/10, Bits=2.41-2.42, Context Length=81922026.06 | 4.1 | |
| QuIP#Backbone=Llama-2-70B, Type=VQ, Bits=2.00, Context Length=20482026.06 | 4.16 | |
| VPTQBackbone=Llama-2-70B, Type=VQ, Bits=2.02-2.08, Context Length=20482026.06 | 4.17 | |
| AQLMBackbone=Llama-2-70B, Type=VQ, Bits=1.97-2.07, Context Length=20482026.06 | 4.19 | |
| EQAT-g64Backbone=Llama-2-70B, Type=UQ, Bits=2.25, Context Length=20482026.06 | 4.52 | |
| ARMORSparsity=2:4+o, Backbone=Llama-2 70B, Context Length=40962025.10 | 4.55 | |
| DenseSparsity=0%, Backbone=Llama-2 13B, Context Length=40962025.10 | 4.57 | |
| AQLM (Drop)Model=MetaLlama 8B, Codebooks=3 × 82026.06 | 4.58 | |
| LiftQuantBackbone=Llama-3-70B, Type=LQ-32/16, Bits=2.01-2.02, Context Length=81922026.06 | 4.69 | |
| GPTQBackbone=Llama-2-70B, Type=UQ, Bits=3.00, Context Length=20482026.06 | 4.82 | |
| GPTQ-g128Backbone=Llama-3-70B, Type=UQ, Bits=3.13, Context Length=81922026.06 | 4.82 | |
| LLaMA2-13BModel=LLaMA2-13B, W bits=-, Compensation Method=-, Quantization Algorithm=GPTQ, Rank=1282024.10 | 4.88 | |
| Baseline (LLaMA2-13B)Model=LLaMA2-13B, Sparsity=None2024.10 | 4.88 | |
| FP16Model=Llama-2-13B2026.03 | 4.88 | |
| FP16Model=L2-13B, Calibration samples=1282026.04 | 4.88 | |
| FP16Model=L2-13B, Bits=16-bit2026.04 | 4.88 | |
| LLaMA2-13BBackbone=LLaMA2-13B, Bit=-, T × D=-, R=-, FLOPs(T)=13.42, Power(J)=61.742026.04 | 4.88 | |
| FP16Backbone=Llama-2-13B, Context Length=20482026.06 | 4.88 | |
| QTIPBackbone=Llama-3-70B, Type=VQ, Bits=2.00, Context Length=81922026.06 | 4.97 | |
| AQLM (Ind.)Model=Qwen 32B, Codebooks=5 × 82026.06 | 4.99 | |
| AQLM (Drop)Model=Qwen 32B, Codebooks=5 × 82026.06 | 4.99 | |
| DbyD (35W)Model=Mistral 7B, Codebooks=5 × 82026.06 | 5 | |
| DbyD (35W)Model=Qwen 32B, Codebooks=5 × 82026.06 | 5.01 | |
| ZeroQuant-V2Model=LLaMA2-13B, W bits=W4, Compensation Method=ZeroQuant-V2, Quantization Algorithm=GPTQ, Rank=1282024.10 | 5.03 | |
| EoRAModel=LLaMA2-13B, W bits=W4, Compensation Method=EoRA, Quantization Algorithm=GPTQ, Rank=1282024.10 | 5.03 | |
| GPTQ+OursModel=L3-70B, Bits=3-bit, Group Size=g1282026.04 | 5.03 | |
| Act-SModel=LLaMA2-13B, W bits=W4, Compensation Method=Act-S, Quantization Algorithm=GPTQ, Rank=1282024.10 | 5.04 | |
| ApiQModel=LLaMA2-13B, W bits=W4, Compensation Method=ApiQ, Quantization Algorithm=GPTQ, Rank=1282024.10 | 5.04 | |
| DbyD (Uni.)Model=Qwen 32B, Codebooks=5 × 82026.06 | 5.04 | |
| GPTQModel=LLaMA2-13B, W bits=W4, Compensation Method=-, Quantization Algorithm=GPTQ, Rank=1282024.10 | 5.06 | |
| AQLM (Ind.)Model=Mistral 7B, Codebooks=4 × 82026.06 | 5.07 | |
| LiftQuantBackbone=Llama-2-13B, Type=LQ-24/8, Bits=3.01-3.02, Context Length=20482026.06 | 5.09 | |
| QuIP#Backbone=Llama-2-13B, Type=VQ, Bits=3.00, Context Length=20482026.06 | 5.1 | |
| DenseSparsity=0%, Backbone=Llama-2 7B, Context Length=40962025.10 | 5.12 | |
| EQAT-g128Backbone=Llama-2-13B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 5.12 | |
| VPTQ#Backbone=Llama-2-13B, Type=VQ, Bits=3.01-3.03, Context Length=20482026.06 | 5.12 | |
| AQLM (Ind.)Model=Qwen 32B, Codebooks=4 × 82026.06 | 5.13 | |
| AQLM (Ind.)Model=Mistral 7B, Codebooks=5 × 82026.06 | 5.14 | |
| AQLM (Drop)Model=Mistral 7B, Codebooks=5 × 82026.06 | 5.14 | |
| GPTQModel=L3-70B, Bits=3-bit, Group Size=g1282026.04 | 5.16 | |
| NoWag-PSparsity=2:4, Backbone=Llama-2 70B, Context Length=40962025.10 | 5.17 | |
| SDLLMBackbone=LLaMA2-13B, Bit=4-1.58, T × D=1 × 16, R=0.215, FLOPs(T)=2.89, Power(J)=2.602026.04 | 5.18 | |
| DbyD (Uni.)Model=Mistral 7B, Codebooks=5 × 82026.06 | 5.19 | |
| WandaSparsity=2:4, Backbone=Llama-2 70B, Context Length=40962025.10 | 5.2 | |
| FP16Model=Qwen2.5-14B2026.03 | 5.29 | |
| Qwen2.5-14BFLOPs(T)=13.53, Power(J)=62.232026.04 | 5.29 | |
| LiftQuantBackbone=Llama-2-13B, Type=LQ-24/10, Bits=2.41-2.42, Context Length=20482026.06 | 5.3 | |
| AWQModel=L3-70B, Bits=3-bit, Group Size=g1282026.04 | 5.36 | |
| EQAT-g64Backbone=Llama-3-70B, Type=UQ, Bits=2.25, Context Length=81922026.06 | 5.36 | |
| QuarotBackbone=Llama-2-13B, Type=UQ, Bits=3.00, Context Length=20482026.06 | 5.37 | |
| AQLM (Ind.)Model=Mistral 7B, Codebooks=3 × 82026.06 | 5.38 | |
| AQLM (Ind.)Model=Qwen 14B, Codebooks=5 × 82026.06 | 5.38 | |
| AQLM (Drop)Model=Qwen 14B, Codebooks=5 × 82026.06 | 5.38 | |
| DbyD (35W)Model=Qwen 32B, Codebooks=4 × 82026.06 | 5.38 | |
| SparseGPTSparsity=2:4, Backbone=Llama-2 70B, Context Length=40962025.10 | 5.39 | |
| GPTAQ+OursModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 5.39 | |
| GPTQ+OursModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 5.42 | |
| GPTAQModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 5.42 | |
| GPTQ-g128Backbone=Llama-2-13B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 5.42 | |
| GPTQModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 5.43 | |
| DbyD (Uni.)Model=Qwen 32B, Codebooks=4 × 82026.06 | 5.43 | |
| DbyD (35W)Model=Mistral 7B, Codebooks=4 × 82026.06 | 5.44 | |
| QTIPBackbone=Llama-2-13B, Type=VQ, Bits=2.00, Context Length=20482026.06 | 5.45 | |
| LLaMA2-7BModel=LLaMA2-7B, W bits=-, Compensation Method=-, Quantization Algorithm=GPTQ, Rank=1282024.10 | 5.47 | |
| Baseline (LLaMA2-7B)Model=LLaMA2-7B, Sparsity=None2024.10 | 5.47 | |
| FP16Model=L2-7B, Calibration samples=1282026.04 | 5.47 | |
| FP16Model=L3-8B, Calibration samples=1282026.04 | 5.47 | |
| FP16Model=L2-7B, Bits=16-bit2026.04 | 5.47 | |
| LLaMA2-7BBackbone=LLaMA2-7B, Bit=-, T × D=-, R=-, FLOPs(T)=6.91, Power(J)=31.772026.04 | 5.47 | |
| FP16Backbone=LLaMA 2 7B2024.12 | 5.47 | |
| FP16Backbone=Llama-2-7B, Context Length=20482026.06 | 5.47 | |
| SDLLMBackbone=LLaMA2-13B, Bit=4-1.58, T × D=1 × 8, R=0.209, FLOPs(T)=1.40, Power(J)=1.262026.04 | 5.48 | |
| AQLM (Ind.)Model=Qwen 32B, Codebooks=3 × 82026.06 | 5.48 | |
| DbyD (Uni.)Model=Qwen 14B, Codebooks=5 × 82026.06 | 5.51 | |
| AQLM (Drop)Model=Qwen 32B, Codebooks=4 × 82026.06 | 5.51 | |
| AWQModel=L2-13B, Bits=3-bit, Group Size=g1282026.04 | 5.53 | |
| DbyD (35W)Model=Qwen 14B, Codebooks=5 × 82026.06 | 5.53 | |
| DenseSparsity=0%, Backbone=Llama-3 8B, Context Length=81922025.10 | 5.54 | |
| FP16Backbone=Llama-3-8B, Context Length=81922026.06 | 5.54 | |
| MixLLMBackbone=LLaMA 2 7B, Quantization=W4.4A82024.12 | 5.55 | |
| VPTQBackbone=Llama-3-70B, Type=VQ, Bits=2.02-2.08, Context Length=81922026.06 | 5.55 | |
| SqueezeLLMBackbone=LLaMA 2 7B, Quantization=W4A16 0.45%2024.12 | 5.57 | |
| OmniQuantBackbone=LLaMA 2 7B, Quantization=W4A162024.12 | 5.58 |