Language Modeling on C4
4.77PerplexityLlama2-7B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Llama2-7BBase Model=Llama2-7B, Watermarking=None2023.11 | 4.77 | — | — | |
| FP16Model=Llama 2-70B, Bits=16, Context Length=40962024.06 | 4.97 | — | — | |
| baselineBackbone=Llama-2-70B, Avg. Num. Bits=162024.01 | 4.97 | — | — | |
| KVQuant-4bit-1%Backbone=Llama-2-70B, Avg. Num. Bits=4.32-4.352024.01 | 4.98 | — | — | |
| FlexGen-4bitBackbone=Llama-2-70B, Avg. Num. Bits=4.312024.01 | 4.99 | — | — | |
| KVQuant-4bitBackbone=Llama-2-70B, Avg. Num. Bits=4.00-4.022024.01 | 4.99 | — | — | |
| KVQuant-3bit-1%Backbone=Llama-2-70B, Avg. Num. Bits=3.32-3.352024.01 | 4.99 | — | — | |
| QTIP (1MAD)Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.06 | 5 | — | — | |
| QTIP (3INST)Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.06 | 5 | — | — | |
| ATOM-4bitBackbone=Llama-2-70B, Avg. Num. Bits=4.162024.01 | 5 | — | — | |
| QuIP#Model=Llama 2-70B, Bits=4, Fine-tuning=false, Context Length=40962024.06 | 5.02 | — | — | |
| QuIP#Model=Llama 2-70B, Bits=4, Fine-tuning=true, Context Length=40962024.06 | 5.02 | — | — | |
| AQLMModel=Llama 2-70B, Bits=4, Fine-tuning=true, Context Length=40962024.06 | 5.03 | — | — | |
| KVQuant-3bitBackbone=Llama-2-70B, Avg. Num. Bits=3.00-3.022024.01 | 5.03 | — | — | |
| nf4Backbone=Llama-2-70B, Avg. Num. Bits=4.00-4.032024.01 | 5.06 | — | — | |
| FlexGen-3bitBackbone=Llama-2-70B, Avg. Num. Bits=3.302024.01 | 5.08 | — | — | |
| KVQuant-2bit-1%Backbone=Llama-2-70B, Avg. Num. Bits=2.32-2.352024.01 | 5.08 | — | — | |
| QTIP (1MAD)Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.06 | 5.09 | — | — | |
| QTIP (3INST)Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.06 | 5.09 | — | — | |
| OriginalPrecision=16-bit2025.03 | 5.12 | — | — | |
| int4Backbone=Llama-2-70B, Avg. Num. Bits=4.00-4.022024.01 | 5.12 | — | — | |
| QuIP#Model=Llama 2-70B, Bits=3, Fine-tuning=true, Context Length=40962024.06 | 5.15 | — | — | |
| ATOM-3bitBackbone=Llama-2-70B, Avg. Num. Bits=3.152024.01 | 5.16 | — | — | |
| AQLMModel=Llama 2-70B, Bits=3, Fine-tuning=true, Context Length=40962024.06 | 5.17 | — | — | |
| QuIP#Model=Llama 2-70B, Bits=3, Fine-tuning=false, Context Length=40962024.06 | 5.2 | — | — | |
| FullPrecisionModel=LLaMA2, Size=70B, W-bits=16.002025.11 | 5.24 | — | — | |
| KVQuant-2bitBackbone=Llama-2-70B, Avg. Num. Bits=2.00-2.022024.01 | 5.3 | — | — | |
| FullPrecisionModel=LLaMA1, Size=65B, W-bits=16.002025.11 | 5.31 | — | — | |
| WATME_dictionaryBase Model=Llama2-7B, Watermarking=WATME_dictionary2023.11 | 5.32 | 98.04 | — | |
| W16A16 BaselineModel=Llama2-13B, #Bits=W16A162026.03 | 5.46 | — | — | |
| FP16 BaselineModel=LLaMA-2-70B, #Bits=W16A162024.02 | 5.52 | — | — | |
| FP16Model=LLaMA-2-70B, Precision=FP162024.06 | 5.52 | — | — | |
| FP16Model=L2-70B, bits=162026.02 | 5.52 | — | — | |
| SNRQ-CModel=L2-70B, bits=4, Q.Time (s)=3893.72026.02 | 5.59 | — | — | |
| GPTQModel=L2-70B, bits=4, Q.Time (s)=3382.32026.02 | 5.6 | — | — | |
| LDLQModel=L2-70B, bits=4, Q.Time (s)=3582.12026.02 | 5.6 | — | — | |
| SNRQ-SModel=L2-70B, bits=4, Q.Time (s)=3713.22026.02 | 5.6 | — | — | |
| AQLMModel=Llama 2-70B, Bits=2, Fine-tuning=true, Context Length=40962024.06 | 5.62 | — | — | |
| W16A16Precision=W16A16, Backbone=LLaMA-1-65B2024.02 | 5.62 | — | — | |
| FP16Model=LLaMA-1-65B, Precision=FP162024.06 | 5.62 | — | — | |
| baselineBackbone=LLaMA-65B, Avg. Num. Bits=162024.01 | 5.62 | — | — | |
| KVQuant-4bit-1%Backbone=LLaMA-65B, Avg. Num. Bits=4.32-4.352024.01 | 5.62 | — | — | |
| GPTAQModel=L2-70B, bits=4, Q.Time (s)=4835.42026.02 | 5.62 | — | — | |
| FullPrecisionModel=LLaMA1, Size=30B, W-bits=16.002025.11 | 5.62 | — | — | |
| FlexGen-4bitBackbone=LLaMA-65B, Avg. Num. Bits=4.312024.01 | 5.63 | — | — | |
| KVQuant-4bitBackbone=LLaMA-65B, Avg. Num. Bits=4.00-4.022024.01 | 5.63 | — | — | |
| KVQuant-3bit-1%Backbone=LLaMA-65B, Avg. Num. Bits=3.32-3.352024.01 | 5.63 | — | — | |
| ATOM-4bitBackbone=LLaMA-65B, Avg. Num. Bits=4.162024.01 | 5.65 | — | — | |
| KVQuant-3bitBackbone=LLaMA-65B, Avg. Num. Bits=3.00-3.022024.01 | 5.68 | — | — | |
| QTIP (3INST)Model=Llama 2-70B, Bits=2, Fine-tuning=false, Context Length=40962024.06 | 5.69 | — | — | |
| nf4Backbone=LLaMA-65B, Avg. Num. Bits=4.00-4.032024.01 | 5.69 | — | — | |
| QTIP (1MAD)Model=Llama 2-70B, Bits=2, Fine-tuning=false, Context Length=40962024.06 | 5.7 | — | — | |
| FlexGen-3bitBackbone=LLaMA-65B, Avg. Num. Bits=3.302024.01 | 5.7 | — | — | |
| KVQuant-2bit-1%Backbone=LLaMA-65B, Avg. Num. Bits=2.32-2.352024.01 | 5.7 | — | — | |
| QuIP#Model=Llama 2-70B, Bits=2, Fine-tuning=true, Context Length=40962024.06 | 5.71 | — | — | |
| baselineBackbone=Mistral-7B, Avg. Num. Bits=162024.01 | 5.71 | — | — | |
| FP16W Quant.=None, Backbone=LLaMA-2-70B2025.11 | 5.71 | — | — | |
| W16A16 BaselineModel=Llama-2-70B, #Bits=W16A162026.03 | 5.71 | — | — | |
| KVQuant-4bit-1%Backbone=Mistral-7B, Avg. Num. Bits=4.32-4.352024.01 | 5.72 | — | — | |
| int4Backbone=LLaMA-65B, Avg. Num. Bits=4.00-4.022024.01 | 5.75 | — | — | |
| FlexGen-4bitBackbone=Mistral-7B, Avg. Num. Bits=4.312024.01 | 5.75 | — | — | |
| KVQuant-4bitBackbone=Mistral-7B, Avg. Num. Bits=4.00-4.022024.01 | 5.75 | — | — | |
| ATOM-4bitBackbone=Mistral-7B, Avg. Num. Bits=4.162024.01 | 5.76 | — | — | |
| KVQuant-3bit-1%Backbone=Mistral-7B, Avg. Num. Bits=3.32-3.352024.01 | 5.76 | — | — | |
| baselineBackbone=Llama-3-70B, Avg. Num. Bits=162024.01 | 5.78 | — | — | |
| KVQuant-4bit-1%Backbone=Llama-3-70B, Avg. Num. Bits=4.32-4.352024.01 | 5.78 | — | — | |
| ATOM-3bitBackbone=LLaMA-65B, Avg. Num. Bits=3.152024.01 | 5.79 | — | — | |
| KVQuant-4bitBackbone=Llama-3-70B, Avg. Num. Bits=4.00-4.022024.01 | 5.8 | — | — | |
| SingleQuantW Quant.=RTN, Backbone=LLaMA-2-70B2025.11 | 5.8 | — | — | |
| KVQuant-3bit-1%Backbone=Llama-3-70B, Avg. Num. Bits=3.32-3.352024.01 | 5.81 | — | — | |
| FlexGen-4bitBackbone=Llama-3-70B, Avg. Num. Bits=4.312024.01 | 5.84 | — | — | |
| DuQuant+LWCModel=LLaMA-2-70B, Precision=W4A42024.06 | 5.85 | — | — | |
| nf4Backbone=Mistral-7B, Avg. Num. Bits=4.00-4.032024.01 | 5.85 | — | — | |
| DuQuantModel=LLaMA-2-70B, Precision=W4A42024.06 | 5.87 | — | — | |
| DuQuantW Quant.=RTN, Backbone=LLaMA-2-70B2025.11 | 5.87 | — | — | |
| SliderQuant+Model=Llama-2-70B, #Bits=W4A42026.03 | 5.87 | — | — | |
| MergeQuantW Quant.=GPTQ, Backbone=LLaMA-2-70B2025.11 | 5.89 | — | — | |
| KVQuant-3bitBackbone=Llama-3-70B, Avg. Num. Bits=3.00-3.022024.01 | 5.9 | — | — | |
| int4Backbone=Mistral-7B, Avg. Num. Bits=4.00-4.022024.01 | 5.91 | — | — | |
| FlatQuantModel=Llama-2-70B, #Bits=W4A42026.03 | 5.91 | — | — | |
| FlexGen-3bitBackbone=Mistral-7B, Avg. Num. Bits=3.302024.01 | 5.92 | — | — | |
| KVQuant-3bitBackbone=Mistral-7B, Avg. Num. Bits=3.00-3.022024.01 | 5.92 | — | — | |
| DuQuant+LWCModel=LLaMA-1-65B, Precision=W4A42024.06 | 5.93 | — | — | |
| SNRQ-SModel=L2-70B, bits=3, Q.Time (s)=3899.82026.02 | 5.94 | — | — | |
| GPTQModel=L2-70B, bits=3, Q.Time (s)=3381.62026.02 | 5.95 | — | — | |
| GPTAQModel=L2-70B, bits=3, Q.Time (s)=4762.22026.02 | 5.95 | — | — | |
| SNRQ-CModel=L2-70B, bits=3, Q.Time (s)=4026.52026.02 | 5.95 | — | — | |
| nf3Backbone=Llama-2-70B, Avg. Num. Bits=3.00-3.032024.01 | 5.96 | — | — | |
| KVQuant-2bitBackbone=LLaMA-65B, Avg. Num. Bits=2.00-2.022024.01 | 5.96 | — | — | |
| KVQuant-2bit-1%Backbone=Llama-3-70B, Avg. Num. Bits=2.32-2.352024.01 | 5.96 | — | — | |
| LDLQModel=L2-70B, bits=3, Q.Time (s)=34162026.02 | 5.96 | — | — | |
| W16A16Precision=W16A16, Backbone=LLaMA-1-30B2024.02 | 5.98 | — | — | |
| FP16Model=LLaMA-1-30B, Precision=FP162024.06 | 5.98 | — | — | |
| baselineBackbone=LLaMA-30B, Avg. Num. Bits=162024.01 | 5.98 | — | — | |
| KVQuant-4bit-1%Backbone=LLaMA-30B, Avg. Num. Bits=4.32-4.352024.01 | 5.99 | — | — | |
| FlexGen-4bitBackbone=LLaMA-30B, Avg. Num. Bits=4.312024.01 | 6 | — | — | |
| KVQuant-4bitBackbone=LLaMA-30B, Avg. Num. Bits=4.00-4.022024.01 | 6 | — | — | |
| KVQuant-3bit-1%Backbone=LLaMA-30B, Avg. Num. Bits=3.32-3.352024.01 | 6 | — | — | |
| QuIP#Model=Llama 2-70B, Bits=2, Fine-tuning=false, Context Length=40962024.06 | 6.01 | — | — | |
| ATOM-4bitBackbone=LLaMA-30B, Avg. Num. Bits=4.162024.01 | 6.02 | — | — |