Language Modeling on WikiText2
2.73PerplexityFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Bits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 2.73 | — | |
| FP16Bits (W-A-KV)=2-4-16, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 2.73 | — | |
| Full precision# Bits=Full precision, Model Backbone=Llama-3 70B2025.06 | 2.86 | — | |
| BaselineBase Model=LLAMA-3 70B, Sequence Length=20482024.03 | 2.86 | — | |
| FP16W-A-KV bits=16-16-16, Backbone=LLaMA-3 70B2025.11 | 2.86 | — | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 2.86 | — | |
| FP16#Bits W-A-KV=3-3-3, Model=LLaMA3-70B2026.05 | 2.86 | — | |
| FP16#Bits W-A-KV=2-4-16, Model=LLaMA3-70B2026.05 | 2.86 | — | |
| FP16Bits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 2.86 | — | |
| FP16Bits (W-A-KV)=2-4-16, Model=LLaMA3-70B, Zero-shot=true2026.05 | 2.86 | — | |
| FloatingPointBackbone=LLaMA-3 70B, W-A-KV Bits=16-16-162026.05 | 2.86 | — | |
| FP16Model=Llama 2-70B, Bits=16, Context Length=40962024.06 | 3.12 | — | |
| FP16Bits=16, BPP=16.00, Model Architecture=Llama 2 70B, Context Length=40962026.05 | 3.12 | — | |
| QTIP (1MAD)Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.06 | 3.16 | — | |
| QTIP (3INST)Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.06 | 3.16 | — | |
| QuIP#Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.06 | 3.18 | — | |
| QuIP#Model=Llama 2-70B, Bits=4, Fine-tuning=true, Context Length=40962024.06 | 3.18 | — | |
| UniQuanF# Bits=4, Scheme=BCQ, Model Backbone=Llama-3 70B2025.06 | 3.19 | — | |
| AQLMModel=Llama 2-70B, Bits=4, Fine-tuning=true, Context Length=40962024.06 | 3.19 | — | |
| OSTQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-16, Distillation=false2026.05 | 3.19 | — | |
| QTIP (1MAD)Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.06 | 3.27 | — | |
| QTIP (3INST)Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.06 | 3.27 | — | |
| FlexRound# Bits=4, Scheme=UQ, Model Backbone=Llama-3 70B2025.06 | 3.31 | — | |
| FP16Model=LLaMA-2-70B, Precision=FP162024.06 | 3.31 | — | |
| FP16Model Scale=70B, Bit-width=2-bit, Group-size=642026.02 | 3.31 | — | |
| FP16Model Scale=70B, Bit-width=3-bit, Group-size=1282026.02 | 3.31 | — | |
| FP16Model Scale=70B, Bit-width=4-bit, Group-size=1282026.02 | 3.31 | — | |
| FP16 BaselineModel=LLaMA-2-70B, #Bits=W16A162024.02 | 3.32 | — | |
| FP16Model=L2-70B, bits=162026.02 | 3.32 | — | |
| W16A16 BaselineModel=Llama-2-70B, #Bits=W16A162026.03 | 3.32 | — | |
| FP16W-A-KV bits=16-16-16, Backbone=LLaMA-2 70B2025.11 | 3.32 | — | |
| FP16Model=LLaMA2-70B, #Bits (W)=16, #Bits (A)=162026.05 | 3.32 | — | |
| FP16#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 3.32 | — | |
| FloatingPointBackbone=LLaMA-2 70B, W-A-KV Bits=16-16-162026.05 | 3.32 | — | |
| FP16Backbone=LLaMA2-70B, #Bits (W)=16, #Bits (A)=162026.06 | 3.32 | — | |
| OmniQuant# Bits=4, Scheme=UQ, Model Backbone=Llama-3 70B2025.06 | 3.34 | — | |
| QuIP#Model=Llama 2-70B, Bits=3, Fine-tuning=true, Context Length=40962024.06 | 3.35 | — | |
| AQLMModel=Llama 2-70B, Bits=3, Fine-tuning=true, Context Length=40962024.06 | 3.36 | — | |
| AstroModel Scale=70B, Bit-width=4-bit, Group-size=1282026.02 | 3.38 | — | |
| OmniQuantModel Scale=70B, Bit-width=4-bit, Group-size=1282026.02 | 3.4 | — | |
| SpecQuantW-A-KV bits=4-16-16, Backbone=LLaMA-2 70B2025.11 | 3.4 | — | |
| INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.05 | 3.4 | — | |
| QuIP#Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.06 | 3.41 | — | |
| QuaRotW-A-KV bits=4-16-16, Backbone=LLaMA-2 70B2025.11 | 3.41 | — | |
| QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.05 | 3.41 | — | |
| OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-16, Distillation=false2026.05 | 3.41 | — | |
| GPTQModel Scale=70B, Bit-width=4-bit, Group-size=1282026.02 | 3.42 | — | |
| MagRModel Scale=70B, Bit-width=4-bit, Group-size=1282026.02 | 3.43 | — | |
| SpinQuantW-A-KV bits=4-16-16, Backbone=LLaMA-2 70B2025.11 | 3.43 | — | |
| SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.05 | 3.43 | — | |
| SNRQ-CModel=L2-70B, bits=4, Q.Time (s)=3893.72026.02 | 3.44 | — | |
| SNRQ-SModel=L2-70B, bits=4, Q.Time (s)=3713.22026.02 | 3.44 | — | |
| GPTQModel=L2-70B, bits=4, Q.Time (s)=3382.32026.02 | 3.45 | — | |
| LDLQModel=L2-70B, bits=4, Q.Time (s)=3582.12026.02 | 3.45 | — | |
| RTNModel Scale=70B, Bit-width=4-bit, Group-size=1282026.02 | 3.46 | — | |
| OursBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 3.46 | — | |
| GPTAQModel=L2-70B, bits=4, Q.Time (s)=4835.42026.02 | 3.47 | — | |
| OmniquantW-A-KV bits=4-16-16, Backbone=LLaMA-2 70B2025.11 | 3.47 | — | |
| SpinQuantW-A-KV bits=4-16-16, Backbone=LLaMA-3 70B2025.11 | 3.49 | — | |
| SpinQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.05 | 3.49 | — | |
| SliderQuant+Model=Llama-2-70B, #Bits=W4A42026.03 | 3.5 | — | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 3.5 | — | |
| INFOQUANTBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.05 | 3.5 | — | |
| BDQ#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 3.52 | — | |
| OursBits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 3.52 | — | |
| FP16Model=LLaMA-1-65B, Precision=FP162024.06 | 3.53 | — | |
| QuaRotW-A-KV bits=4-16-16, Backbone=LLaMA-3 70B2025.11 | 3.53 | — | |
| SpecQuantW-A-KV bits=4-16-16, Backbone=LLaMA-3 70B2025.11 | 3.53 | — | |
| QuaRotBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.05 | 3.53 | — | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 3.54 | — | |
| FlatQuantModel=Llama-2-70B, #Bits=W4A42026.03 | 3.55 | — | |
| OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-16, Distillation=true2026.05 | 3.57 | — | |
| OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-4, Distillation=true2026.05 | 3.59 | — | |
| SpecQuantW-A-KV bits=4-4-4, Backbone=LLaMA-2 70B2025.11 | 3.6 | — | |
| SpinQuantW-A-KV bits=4-4-4, Backbone=LLaMA-2 70B2025.11 | 3.61 | — | |
| SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.05 | 3.61 | — | |
| INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.05 | 3.62 | — | |
| SpecQuantW-A-KV bits=4-4-16, Backbone=LLaMA-2 70B2025.11 | 3.63 | — | |
| INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.05 | 3.64 | — | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=DeepSeek-R1-Distill LLaMA-70B, Zero-shot=true2026.05 | 3.65 | — | |
| SpinQuantW-A-KV bits=4-4-16, Backbone=LLaMA-2 70B2025.11 | 3.68 | — | |
| SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.05 | 3.68 | — | |
| AstroModel Scale=70B, Bit-width=3-bit, Group-size=1282026.02 | 3.69 | — | |
| SpinQuantModel=Llama-2-70B, #Bits=W4A42026.03 | 3.7 | — | |
| SpinQuant#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 3.7 | — | |
| MagRModel Scale=70B, Bit-width=3-bit, Group-size=1282026.02 | 3.71 | — | |
| DuQuant+LWCModel=LLaMA-2-70B, Precision=W4A42024.06 | 3.76 | — | |
| FlatQuant#Bits W-A-KV=4-4-4, Model=LLaMA3-70B2026.05 | 3.77 | — | |
| FlatQuantBits (W-A-KV)=4-4-4, Model=LLaMA3-70B, Zero-shot=true2026.05 | 3.77 | — | |
| OmniQuantModel Scale=70B, Bit-width=3-bit, Group-size=1282026.02 | 3.78 | — | |
| QuaRotW-A-KV bits=4-4-16, Backbone=LLaMA-2 70B2025.11 | 3.78 | — | |
| QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.05 | 3.78 | — | |
| DuQuantModel=LLaMA-2-70B, Precision=W4A42024.06 | 3.79 | — | |
| QuaRotModel=Llama-2-70B, #Bits=W4A42026.03 | 3.79 | — | |
| QuaRot#Bits W-A-KV=4-4-4, Model=LLaMA2-70B2026.05 | 3.79 | — | |
| QuaRotW-A-KV bits=4-4-4, Backbone=LLaMA-2 70B2025.11 | 3.8 | — | |
| QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.05 | 3.8 | — | |
| LoRAW Bits=16, Model=LLaMA 65B2023.05 | 3.82 | — | |
| AQLMModel=Llama 2-70B, Bits=2, Fine-tuning=true, Context Length=40962024.06 | 3.83 | — | |
| FP16Model=Mixtral-8x7B, Bit=162026.01 | 3.84 | — |