List your company
ResearchFormerly WizWand

Language Modeling on C4

4.77PerplexityLlama2-7B
4.725.05755.3955.7325Nov 16, 2023Apr 7, 2024Aug 29, 2024Jan 19, 2025Jun 12, 2025Nov 2, 2025Mar 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Llama2-7BBase Model=Llama2-7B, Watermarking=None2023.114.77——
FP16Model=Llama 2-70B, Bits=16, Context Length=40962024.064.97——
baselineBackbone=Llama-2-70B, Avg. Num. Bits=162024.014.97——
KVQuant-4bit-1%Backbone=Llama-2-70B, Avg. Num. Bits=4.32-4.352024.014.98——
FlexGen-4bitBackbone=Llama-2-70B, Avg. Num. Bits=4.312024.014.99——
KVQuant-4bitBackbone=Llama-2-70B, Avg. Num. Bits=4.00-4.022024.014.99——
KVQuant-3bit-1%Backbone=Llama-2-70B, Avg. Num. Bits=3.32-3.352024.014.99——
QTIP (1MAD)Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.065——
QTIP (3INST)Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.065——
ATOM-4bitBackbone=Llama-2-70B, Avg. Num. Bits=4.162024.015——
QuIP#Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.065.02——
QuIP#Model=Llama 2-70B, Bits=4, Fine-tuning=true, Context Length=40962024.065.02——
AQLMModel=Llama 2-70B, Bits=4, Fine-tuning=true, Context Length=40962024.065.03——
KVQuant-3bitBackbone=Llama-2-70B, Avg. Num. Bits=3.00-3.022024.015.03——
nf4Backbone=Llama-2-70B, Avg. Num. Bits=4.00-4.032024.015.06——
FlexGen-3bitBackbone=Llama-2-70B, Avg. Num. Bits=3.302024.015.08——
KVQuant-2bit-1%Backbone=Llama-2-70B, Avg. Num. Bits=2.32-2.352024.015.08——
QTIP (1MAD)Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.065.09——
QTIP (3INST)Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.065.09——
OriginalPrecision=16-bit2025.035.12——
int4Backbone=Llama-2-70B, Avg. Num. Bits=4.00-4.022024.015.12——
QuIP#Model=Llama 2-70B, Bits=3, Fine-tuning=true, Context Length=40962024.065.15——
ATOM-3bitBackbone=Llama-2-70B, Avg. Num. Bits=3.152024.015.16——
AQLMModel=Llama 2-70B, Bits=3, Fine-tuning=true, Context Length=40962024.065.17——
QuIP#Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.065.2——
FullPrecisionModel=LLaMA2, Size=70B, W-bits=16.002025.115.24——
KVQuant-2bitBackbone=Llama-2-70B, Avg. Num. Bits=2.00-2.022024.015.3——
FullPrecisionModel=LLaMA1, Size=65B, W-bits=16.002025.115.31——
WATME_dictionaryBase Model=Llama2-7B, Watermarking=WATME_dictionary2023.115.3298.04—
W16A16 BaselineModel=Llama2-13B, #Bits=W16A162026.035.46——
FP16 BaselineModel=LLaMA-2-70B, #Bits=W16A162024.025.52——
FP16Model=LLaMA-2-70B, Precision=FP162024.065.52——
FP16Model=L2-70B, bits=162026.025.52——
SNRQ-CModel=L2-70B, bits=4, Q.Time (s)=3893.72026.025.59——
GPTQModel=L2-70B, bits=4, Q.Time (s)=3382.32026.025.6——
LDLQModel=L2-70B, bits=4, Q.Time (s)=3582.12026.025.6——
SNRQ-SModel=L2-70B, bits=4, Q.Time (s)=3713.22026.025.6——
AQLMModel=Llama 2-70B, Bits=2, Fine-tuning=true, Context Length=40962024.065.62——
W16A16Precision=W16A16, Backbone=LLaMA-1-65B2024.025.62——
FP16Model=LLaMA-1-65B, Precision=FP162024.065.62——
baselineBackbone=LLaMA-65B, Avg. Num. Bits=162024.015.62——
KVQuant-4bit-1%Backbone=LLaMA-65B, Avg. Num. Bits=4.32-4.352024.015.62——
GPTAQModel=L2-70B, bits=4, Q.Time (s)=4835.42026.025.62——
FullPrecisionModel=LLaMA1, Size=30B, W-bits=16.002025.115.62——
FlexGen-4bitBackbone=LLaMA-65B, Avg. Num. Bits=4.312024.015.63——
KVQuant-4bitBackbone=LLaMA-65B, Avg. Num. Bits=4.00-4.022024.015.63——
KVQuant-3bit-1%Backbone=LLaMA-65B, Avg. Num. Bits=3.32-3.352024.015.63——
ATOM-4bitBackbone=LLaMA-65B, Avg. Num. Bits=4.162024.015.65——
KVQuant-3bitBackbone=LLaMA-65B, Avg. Num. Bits=3.00-3.022024.015.68——
QTIP (3INST)Model=Llama 2-70B, Bits=2, Fine-tuning=false, Context Length=40962024.065.69——
nf4Backbone=LLaMA-65B, Avg. Num. Bits=4.00-4.032024.015.69——
QTIP (1MAD)Model=Llama 2-70B, Bits=2, Fine-tuning=false, Context Length=40962024.065.7——
FlexGen-3bitBackbone=LLaMA-65B, Avg. Num. Bits=3.302024.015.7——
KVQuant-2bit-1%Backbone=LLaMA-65B, Avg. Num. Bits=2.32-2.352024.015.7——
QuIP#Model=Llama 2-70B, Bits=2, Fine-tuning=true, Context Length=40962024.065.71——
baselineBackbone=Mistral-7B, Avg. Num. Bits=162024.015.71——
FP16W Quant.=None, Backbone=LLaMA-2-70B2025.115.71——
W16A16 BaselineModel=Llama-2-70B, #Bits=W16A162026.035.71——
KVQuant-4bit-1%Backbone=Mistral-7B, Avg. Num. Bits=4.32-4.352024.015.72——
int4Backbone=LLaMA-65B, Avg. Num. Bits=4.00-4.022024.015.75——
FlexGen-4bitBackbone=Mistral-7B, Avg. Num. Bits=4.312024.015.75——
KVQuant-4bitBackbone=Mistral-7B, Avg. Num. Bits=4.00-4.022024.015.75——
ATOM-4bitBackbone=Mistral-7B, Avg. Num. Bits=4.162024.015.76——
KVQuant-3bit-1%Backbone=Mistral-7B, Avg. Num. Bits=3.32-3.352024.015.76——
baselineBackbone=Llama-3-70B, Avg. Num. Bits=162024.015.78——
KVQuant-4bit-1%Backbone=Llama-3-70B, Avg. Num. Bits=4.32-4.352024.015.78——
ATOM-3bitBackbone=LLaMA-65B, Avg. Num. Bits=3.152024.015.79——
KVQuant-4bitBackbone=Llama-3-70B, Avg. Num. Bits=4.00-4.022024.015.8——
SingleQuantW Quant.=RTN, Backbone=LLaMA-2-70B2025.115.8——
KVQuant-3bit-1%Backbone=Llama-3-70B, Avg. Num. Bits=3.32-3.352024.015.81——
FlexGen-4bitBackbone=Llama-3-70B, Avg. Num. Bits=4.312024.015.84——
DuQuant+LWCModel=LLaMA-2-70B, Precision=W4A42024.065.85——
nf4Backbone=Mistral-7B, Avg. Num. Bits=4.00-4.032024.015.85——
DuQuantModel=LLaMA-2-70B, Precision=W4A42024.065.87——
DuQuantW Quant.=RTN, Backbone=LLaMA-2-70B2025.115.87——
SliderQuant+Model=Llama-2-70B, #Bits=W4A42026.035.87——
MergeQuantW Quant.=GPTQ, Backbone=LLaMA-2-70B2025.115.89——
KVQuant-3bitBackbone=Llama-3-70B, Avg. Num. Bits=3.00-3.022024.015.9——
int4Backbone=Mistral-7B, Avg. Num. Bits=4.00-4.022024.015.91——
FlatQuantModel=Llama-2-70B, #Bits=W4A42026.035.91——
FlexGen-3bitBackbone=Mistral-7B, Avg. Num. Bits=3.302024.015.92——
KVQuant-3bitBackbone=Mistral-7B, Avg. Num. Bits=3.00-3.022024.015.92——
DuQuant+LWCModel=LLaMA-1-65B, Precision=W4A42024.065.93——
SNRQ-SModel=L2-70B, bits=3, Q.Time (s)=3899.82026.025.94——
GPTQModel=L2-70B, bits=3, Q.Time (s)=3381.62026.025.95——
GPTAQModel=L2-70B, bits=3, Q.Time (s)=4762.22026.025.95——
SNRQ-CModel=L2-70B, bits=3, Q.Time (s)=4026.52026.025.95——
nf3Backbone=Llama-2-70B, Avg. Num. Bits=3.00-3.032024.015.96——
KVQuant-2bitBackbone=LLaMA-65B, Avg. Num. Bits=2.00-2.022024.015.96——
KVQuant-2bit-1%Backbone=Llama-3-70B, Avg. Num. Bits=2.32-2.352024.015.96——
LDLQModel=L2-70B, bits=3, Q.Time (s)=34162026.025.96——
W16A16Precision=W16A16, Backbone=LLaMA-1-30B2024.025.98——
FP16Model=LLaMA-1-30B, Precision=FP162024.065.98——
baselineBackbone=LLaMA-30B, Avg. Num. Bits=162024.015.98——
KVQuant-4bit-1%Backbone=LLaMA-30B, Avg. Num. Bits=4.32-4.352024.015.99——
FlexGen-4bitBackbone=LLaMA-30B, Avg. Num. Bits=4.312024.016——
KVQuant-4bitBackbone=LLaMA-30B, Avg. Num. Bits=4.00-4.022024.016——
KVQuant-3bit-1%Backbone=LLaMA-30B, Avg. Num. Bits=3.32-3.352024.016——
QuIP#Model=Llama 2-70B, Bits=2, Fine-tuning=false, Context Length=40962024.066.01——
ATOM-4bitBackbone=LLaMA-30B, Avg. Num. Bits=4.162024.016.02——