Language Modeling on C4 (PPL)
5.52PerplexityFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Backbone=Llama-2-70B, Context Length=20482026.06 | 5.52 | |
| FP16Avg. Bit=16, Backbone=LLaMA2-70B2024.10 | 5.52 | |
| CMPQAvg. Bit=4, Backbone=LLaMA2-70B2024.10 | 5.57 | |
| AWQAvg. Bit=4, Backbone=LLaMA2-70B2024.10 | 5.58 | |
| VPTQ#Backbone=Llama-2-70B, Type=VQ, Bits=3.01-3.03, Context Length=20482026.06 | 5.67 | |
| QuIP#Backbone=Llama-2-70B, Type=VQ, Bits=3.00, Context Length=20482026.06 | 5.67 | |
| LiftQuantBackbone=Llama-2-70B, Type=LQ-24/8, Bits=3.01-3.02, Context Length=20482026.06 | 5.67 | |
| GPTQAvg. Bit=4, Backbone=LLaMA2-70B2024.10 | 5.7 | |
| EQAT-g128Backbone=Llama-2-70B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 5.71 | |
| FP16Backbone=Llama-3-70B, Context Length=81922026.06 | 5.78 | |
| CMPQAvg. Bit=3, Backbone=LLaMA2-70B2024.10 | 5.78 | |
| RTNAvg. Bit=4, Backbone=LLaMA2-70B2024.10 | 5.8 | |
| AWQAvg. Bit=3, Backbone=LLaMA2-70B2024.10 | 5.81 | |
| LiftQuantBackbone=Llama-2-70B, Type=LQ-24/10, Bits=2.41-2.42, Context Length=20482026.06 | 5.83 | |
| GPTQ-g128Backbone=Llama-2-70B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 5.85 | |
| QuIPAvg. Bit=4, Backbone=LLaMA2-70B2024.10 | 5.86 | |
| QTIPBackbone=Llama-2-70B, Type=VQ, Bits=2.00, Context Length=20482026.06 | 5.93 | |
| LiftQuantBackbone=Llama-2-70B, Type=LQ-32/16, Bits=2.01-2.02, Context Length=20482026.06 | 6.06 | |
| QuIP#Backbone=Llama-2-70B, Type=VQ, Bits=2.00, Context Length=20482026.06 | 6.12 | |
| AQLMBackbone=Llama-2-70B, Type=VQ, Bits=1.97-2.07, Context Length=20482026.06 | 6.13 | |
| VPTQBackbone=Llama-2-70B, Type=VQ, Bits=2.02-2.08, Context Length=20482026.06 | 6.13 | |
| QuIPAvg. Bit=3, Backbone=LLaMA2-70B2024.10 | 6.14 | |
| QuIP#Backbone=Llama-3-70B, Type=VQ, Bits=3.00, Context Length=81922026.06 | 6.18 | |
| LiftQuantBackbone=Llama-3-70B, Type=LQ-24/8, Bits=3.01-3.02, Context Length=81922026.06 | 6.18 | |
| EQAT-g128Backbone=Llama-3-70B, Type=UQ, Bits=3.13, Context Length=81922026.06 | 6.3 | |
| EQAT-g64Backbone=Llama-2-70B, Type=UQ, Bits=2.25, Context Length=20482026.06 | 6.38 | |
| BF16Algorithm=BF16, Bits=16, Backbone=Llama 3.1 70B Instruct2025.05 | 6.46 | |
| BF16Base Model=Llama 3.1 70B Instruct, Bits=162025.05 | 6.46 | |
| FP16Backbone=Llama-2-13B, Context Length=20482026.06 | 6.47 | |
| LiftQuantBackbone=Llama-3-70B, Type=LQ-24/10, Bits=2.41-2.42, Context Length=81922026.06 | 6.47 | |
| FP16Backbone=2-132026.06 | 6.47 | |
| YAQA-BAlgorithm=YAQA-B, Bits=4, Backbone=Llama 3.1 70B Instruct2025.05 | 6.51 | |
| YAQA-ABase Model=Llama 3.1 70B Instruct, Bits=4, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 6.52 | |
| YAQA-BBase Model=Llama 3.1 70B Instruct, Bits=4, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 6.52 | |
| LDLQBase Model=Llama 3.1 70B Instruct, Bits=4, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 6.54 | |
| GPTQBackbone=Llama-2-70B, Type=UQ, Bits=3.00, Context Length=20482026.06 | 6.57 | |
| YAQA-BBase Model=Llama 3.1 70B Instruct, Bits=3, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 6.61 | |
| YAQA-ABase Model=Llama 3.1 70B Instruct, Bits=3, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 6.63 | |
| LDLQBase Model=Llama 3.1 70B Instruct, Bits=3, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 6.64 | |
| GPTQAvg. Bit=3, Backbone=LLaMA2-70B2024.10 | 6.69 | |
| VPTQ#Backbone=Llama-2-13B, Type=VQ, Bits=3.01-3.03, Context Length=20482026.06 | 6.7 | |
| LiftQuantBackbone=Llama-2-13B, Type=LQ-24/8, Bits=3.01-3.02, Context Length=20482026.06 | 6.71 | |
| QuIP#Backbone=Llama-2-13B, Type=VQ, Bits=3.00, Context Length=20482026.06 | 6.72 | |
| LiftQuantBackbone=Llama-3-70B, Type=LQ-32/16, Bits=2.01-2.02, Context Length=81922026.06 | 6.73 | |
| EQAT-g128Backbone=Llama-2-13B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 6.73 | |
| QTIPBackbone=Llama-3-70B, Type=VQ, Bits=2.00, Context Length=81922026.06 | 6.8 | |
| FP16Backbone=Llama-2-7B, Context Length=20482026.06 | 6.97 | |
| FP16Backbone=2-72026.06 | 6.97 | |
| LiftQuantBackbone=Llama-2-13B, Type=LQ-24/10, Bits=2.41-2.42, Context Length=20482026.06 | 6.98 | |
| GPTQ-g128Backbone=Llama-2-13B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 7 | |
| YAQA-BBase Model=Llama 3.1 70B Instruct, Bits=2, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 7.07 | |
| FP16Backbone=Llama-3-8B, Context Length=81922026.06 | 7.1 | |
| YAQA-ABase Model=Llama 3.1 70B Instruct, Bits=2, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 7.1 | |
| QTIPBackbone=Llama-2-13B, Type=VQ, Bits=2.00, Context Length=20482026.06 | 7.16 | |
| LDLQBase Model=Llama 3.1 70B Instruct, Bits=2, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 7.16 | |
| VPTQBackbone=Llama-3-70B, Type=VQ, Bits=2.02-2.08, Context Length=81922026.06 | 7.3 | |
| LiftQuantBackbone=Llama-2-7B, Type=LQ-24/8, Bits=3.01-3.02, Context Length=20482026.06 | 7.31 | |
| QuIP#Backbone=Llama-2-7B, Type=VQ, Bits=3.00, Context Length=20482026.06 | 7.32 | |
| VPTQ#Backbone=Llama-2-7B, Type=VQ, Bits=3.01-3.03, Context Length=20482026.06 | 7.33 | |
| OriginalCompression Ratio=0%2024.11 | 7.34 | |
| LiftQuantBackbone=Llama-2-13B, Type=LQ-32/16, Bits=2.01-2.02, Context Length=20482026.06 | 7.34 | |
| EQAT-g128Backbone=Llama-2-7B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 7.34 | |
| YAQA-BAlgorithm=YAQA-B, Bits=2, Backbone=Llama 3.1 70B Instruct2025.05 | 7.34 | |
| Base ModelParameter Rate=100%2026.06 | 7.34 | |
| BaselineRatio=0.0, Backbone=LLaMA-7B, Fine-tuning=false2026.07 | 7.34 | |
| GPTQ-g128Backbone=Llama-3-70B, Type=UQ, Bits=3.13, Context Length=81922026.06 | 7.37 | |
| VPTQBackbone=Llama-2-13B, Type=VQ, Bits=2.02-2.08, Context Length=20482026.06 | 7.41 | |
| EQAT-g64Backbone=Llama-3-70B, Type=UQ, Bits=2.25, Context Length=81922026.06 | 7.43 | |
| AQLMBackbone=Llama-2-13B, Type=VQ, Bits=1.97-2.07, Context Length=20482026.06 | 7.44 | |
| QuIP#Backbone=Llama-2-13B, Type=VQ, Bits=2.00, Context Length=20482026.06 | 7.45 | |
| QuIP#Backbone=Llama-3-70B, Type=VQ, Bits=2.00, Context Length=81922026.06 | 7.46 | |
| NeUQISetting=W4A4, Backbone=LLaMA 2 7B, Hadamard Transform=true2025.05 | 7.57 | |
| DbyD (35W)Model=Mistral 7B, Codebooks=5 × 82026.06 | 7.58 | |
| MagRSetting=W4A4, Backbone=LLaMA 2 7B, Hadamard Transform=true2025.05 | 7.59 | |
| EQAT-g64Backbone=Llama-2-13B, Type=UQ, Bits=2.25, Context Length=20482026.06 | 7.59 | |
| OmniQ-g64Backbone=Llama-2-70B, Type=UQ, Bits=2.25, Context Length=20482026.06 | 7.68 | |
| LiftQuantBackbone=Llama-2-7B, Type=LQ-24/10, Bits=2.41-2.42, Context Length=20482026.06 | 7.7 | |
| LiftQuantBackbone=Llama-3-8B, Type=LQ-24/8, Bits=3.01-3.02, Context Length=81922026.06 | 7.7 | |
| DbyD (Uni.)Model=Mistral 7B, Codebooks=5 × 82026.06 | 7.7 | |
| QuIP#Backbone=Llama-3-8B, Type=VQ, Bits=3.00, Context Length=81922026.06 | 7.71 | |
| GPTAQSetting=W4A4, Backbone=LLaMA 2 7B, Hadamard Transform=true2025.05 | 7.73 | |
| AQLM (Ind.)Model=Mistral 7B, Codebooks=4 × 82026.06 | 7.73 | |
| GPTQSetting=W4A4, Backbone=LLaMA 2 7B, Hadamard Transform=true2025.05 | 7.75 | |
| AQLM (Drop)Model=MetaLlama 8B, Codebooks=3 × 82026.06 | 7.78 | |
| EQAT-g128Backbone=Llama-3-8B, Type=UQ, Bits=3.13, Context Length=81922026.06 | 7.79 | |
| GPTQ-g128Backbone=Llama-2-7B, Type=UQ, Bits=3.13, Context Length=20482026.06 | 7.89 | |
| AQLM (Ind.)Model=Mistral 7B, Codebooks=5 × 82026.06 | 7.93 | |
| AQLM (Drop)Model=Mistral 7B, Codebooks=5 × 82026.06 | 7.93 | |
| QTIPBackbone=Llama-2-7B, Type=VQ, Bits=2.00, Context Length=20482026.06 | 7.94 | |
| GPTQBackbone=Llama-2-13B, Type=UQ, Bits=3.00, Context Length=20482026.06 | 8.02 | |
| BF16Algorithm=BF16, Bits=16, Backbone=Llama 3.1 8B Instruct2025.05 | 8.02 | |
| BF16Bits=16, Finetuning=none, Incoherence processing=true, Backbone=Llama 3.1 8B Instruct2025.05 | 8.02 | |
| BF16Base Model=Llama 3.1 8B Instruct, Bits=162025.05 | 8.02 | |
| DbyD (35W)Model=Mistral 7B, Codebooks=4 × 82026.06 | 8.08 | |
| FiPSCompression Ratio=20%2024.11 | 8.1 | |
| YAQA-ABase Model=Llama 3.1 8B Instruct, Bits=4, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 8.1 | |
| YAQA-BBase Model=Llama 3.1 8B Instruct, Bits=4, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 8.11 | |
| YAQA-BAlgorithm=YAQA-B, Bits=4, Backbone=Llama 3.1 8B Instruct2025.05 | 8.12 | |
| LDLQBase Model=Llama 3.1 8B Instruct, Bits=4, Quantizer=QTIP, Recovery Finetuning=true2025.05 | 8.13 | |
| LiftQuantBackbone=Llama-2-7B, Type=LQ-32/16, Bits=2.01-2.02, Context Length=20482026.06 | 8.21 |