Language Modeling on C4 (test)
4.97PerplexityFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Bits=16.0, Model=Llama-2 70B, Context Length=40962024.06 | 4.97 | |
| QTIPBits=4.0, Model=Llama-2 70B, Context Length=40962024.06 | 5 | |
| QuIP#Bits=4.0, Model=Llama-2 70B, Context Length=40962024.06 | 5.02 | |
| AQLMBits=4.0, Model=Llama-2 70B, Context Length=40962024.06 | 5.03 | |
| QTIPBits=3.0, Model=Llama-2 70B, Context Length=40962024.06 | 5.08 | |
| QuIP#Bits=3.0, Model=Llama-2 70B, Context Length=40962024.06 | 5.15 | |
| AQLMBits=3.0, Model=Llama-2 70B, Context Length=40962024.06 | 5.17 | |
| QTIPBits=2.0, Model=Llama-2 70B, Context Length=40962024.06 | 5.48 | |
| DuQuant#Bit=W6A6, Backbone=LLaMA1-30B2026.04 | 5.49 | |
| FP16Bits=16.0, Model=Llama-1 65B, Context Length=20482024.06 | 5.62 | |
| AQLMBits=2.0, Model=Llama-2 70B, Context Length=40962024.06 | 5.62 | |
| QTIPBits=4.0, Model=Llama-1 65B, Context Length=20482024.06 | 5.64 | |
| QuIP#Bits=4.0, Model=Llama-1 65B, Context Length=20482024.06 | 5.66 | |
| QTIPBits=3.0, Model=Llama-1 65B, Context Length=20482024.06 | 5.71 | |
| QuIP#Bits=2.0, Model=Llama-2 70B, Context Length=40962024.06 | 5.71 | |
| FP16Bits Per FPN=16, Backbone=Mistral-7b2024.05 | 5.71 | |
| KVQuant-4b-1%Bits Per FPN=4.32-4.35, Backbone=Mistral-7b2024.05 | 5.72 | |
| CQ-2c8bBits Per FPN=4.00, Backbone=Mistral-7b2024.05 | 5.74 | |
| KVQuant-4bBits Per FPN=4.00-4.02, Backbone=Mistral-7b2024.05 | 5.75 | |
| INT4-gs128Bits Per FPN=4.16, Backbone=Mistral-7b2024.05 | 5.76 | |
| QuIP#Bits=3.0, Model=Llama-1 65B, Context Length=20482024.06 | 5.77 | |
| NF4-gs128Bits Per FPN=4.16, Backbone=Mistral-7b2024.05 | 5.77 | |
| NF4Bits Per FPN=4.00-4.01, Backbone=Mistral-7b2024.05 | 5.85 | |
| INT4Bits Per FPN=4.00-4.01, Backbone=Mistral-7b2024.05 | 5.91 | |
| FP16#Bit=FP16, Backbone=LLaMA1-30B2026.04 | 5.92 | |
| RUQuant#Bit=W6A6, Backbone=LLaMA1-30B2026.04 | 5.94 | |
| FP16Bits=16.0, Model=Llama-1 30B, Context Length=20482024.06 | 5.98 | |
| QTIPBits=4.0, Model=Llama-1 30B, Context Length=20482024.06 | 6.01 | |
| QuIP#Bits=4.0, Model=Llama-1 30B, Context Length=20482024.06 | 6.03 | |
| FP16Bits=16.0, Model=Llama-2 13B, Context Length=40962024.06 | 6.05 | |
| FP16Bits Per FPN=16, Backbone=LLaMA-2-13b2024.05 | 6.05 | |
| KVQuant-4b-1%Bits Per FPN=4.32-4.35, Backbone=LLaMA-2-13b2024.05 | 6.06 | |
| QTIPBits=2.0, Model=Llama-1 65B, Context Length=20482024.06 | 6.08 | |
| KVQuant-2b-1%Bits Per FPN=2.32-2.35, Backbone=Mistral-7b2024.05 | 6.08 | |
| QTIPBits=4.0, Model=Llama-2 13B, Context Length=40962024.06 | 6.09 | |
| QTIPBits=3.0, Model=Llama-1 30B, Context Length=20482024.06 | 6.09 | |
| CQ-2c8bBits Per FPN=4.00, Backbone=LLaMA-2-13b2024.05 | 6.09 | |
| KVQuant-4bBits Per FPN=4.00-4.02, Backbone=LLaMA-2-13b2024.05 | 6.11 | |
| QuIP#Bits=4.0, Model=Llama-2 13B, Context Length=40962024.06 | 6.13 | |
| FP16Model Family=LLaMA, Model Scale=30B, Precision=FP162024.02 | 6.131 | |
| AQLMBits=4.0, Model=Llama-2 13B, Context Length=40962024.06 | 6.14 | |
| QuIP#Bits=3.0, Model=Llama-1 30B, Context Length=20482024.06 | 6.17 | |
| CQ-4c8bBits Per FPN=2.00, Backbone=Mistral-7b2024.05 | 6.17 | |
| QuIP#Bits=2.0, Model=Llama-1 65B, Context Length=20482024.06 | 6.19 | |
| INT4-gs128Bits Per FPN=4.16, Backbone=LLaMA-2-13b2024.05 | 6.2 | |
| NF4-gs128Bits Per FPN=4.16, Backbone=LLaMA-2-13b2024.05 | 6.2 | |
| QTIPBits=3.0, Model=Llama-2 13B, Context Length=40962024.06 | 6.22 | |
| RUQuant#Bit=W4A4, Backbone=LLaMA1-30B2026.04 | 6.25 | |
| aespaModel Family=LLaMA, Model Scale=30B, Precision=INT42024.02 | 6.256 | |
| DuQuant#Bit=W4A4, Backbone=LLaMA1-30B2026.04 | 6.26 | |
| OmniQuantModel Family=LLaMA, Model Scale=30B, Precision=INT42024.02 | 6.269 | |
| AffineQuantModel Family=LLaMA, Model Scale=30B, Precision=INT42024.02 | 6.27 | |
| AQLMBits=3.0, Model=Llama-2 13B, Context Length=40962024.06 | 6.33 | |
| QuIP#Bits=3.0, Model=Llama-2 13B, Context Length=40962024.06 | 6.35 | |
| KVQuant-2b-1%Bits Per FPN=2.32-2.35, Backbone=LLaMA-2-13b2024.05 | 6.38 | |
| FP16#Bit=FP16, Backbone=LLaMA2-13B2026.04 | 6.41 | |
| RUQuant#Bit=W6A6, Backbone=LLaMA2-13B2026.04 | 6.43 | |
| DuQuant#Bit=W6A6, Backbone=LLaMA2-13B2026.04 | 6.44 | |
| NF4Bits Per FPN=4.00-4.01, Backbone=LLaMA-2-13b2024.05 | 6.45 | |
| FP16#Bit=FP16, Backbone=LLaMA1-13B2026.04 | 6.49 | |
| RUQuant#Bit=W6A6, Backbone=LLaMA1-13B2026.04 | 6.51 | |
| CQ-4c8bBits Per FPN=2.00, Backbone=LLaMA-2-13b2024.05 | 6.52 | |
| DuQuant#Bit=W6A6, Backbone=LLaMA1-13B2026.04 | 6.52 | |
| QTIPBits=2.0, Model=Llama-1 30B, Context Length=20482024.06 | 6.56 | |
| INT4Bits Per FPN=4.00-4.01, Backbone=LLaMA-2-13b2024.05 | 6.59 | |
| FP16Bits=16.0, Model=Llama-1 13B, Context Length=20482024.06 | 6.61 | |
| FP16Bits Per FPN=16, Backbone=LLaMA-13b2024.05 | 6.61 | |
| KVQuant-4b-1%Bits Per FPN=4.32-4.35, Backbone=LLaMA-13b2024.05 | 6.62 | |
| FP16Bits=16.0, Model=Llama-2 7B, Context Length=40962024.06 | 6.63 | |
| FP16Bits Per FPN=16, Backbone=LLama-2-7b2024.05 | 6.63 | |
| CQ-2c8bBits Per FPN=4.00, Backbone=LLaMA-13b2024.05 | 6.64 | |
| QTIPBits=4.0, Model=Llama-1 13B, Context Length=20482024.06 | 6.65 | |
| KVQuant-4bBits Per FPN=4.00-4.02, Backbone=LLaMA-13b2024.05 | 6.65 | |
| KVQuant-4b-1%Bits Per FPN=4.32-4.35, Backbone=LLama-2-7b2024.05 | 6.65 | |
| NF4-gs128Bits Per FPN=4.16, Backbone=LLaMA-13b2024.05 | 6.66 | |
| QuIP#Bits=4.0, Model=Llama-1 13B, Context Length=20482024.06 | 6.67 | |
| INT4-gs128Bits Per FPN=4.16, Backbone=LLaMA-13b2024.05 | 6.67 | |
| CQ-2c8bBits Per FPN=4.00, Backbone=LLama-2-7b2024.05 | 6.67 | |
| QTIPBits=4.0, Model=Llama-2 7B, Context Length=40962024.06 | 6.69 | |
| KVQuant-4bBits Per FPN=4.00-4.02, Backbone=LLama-2-7b2024.05 | 6.7 | |
| QuIP#Bits=2.0, Model=Llama-1 30B, Context Length=20482024.06 | 6.71 | |
| FP16Model Family=LLaMA2, Model Scale=13B, Precision=FP162024.02 | 6.727 | |
| QTIPBits=3.0, Model=Llama-1 13B, Context Length=20482024.06 | 6.74 | |
| NF4Bits Per FPN=4.00-4.01, Backbone=LLaMA-13b2024.05 | 6.74 | |
| AQLMBits=4.0, Model=Llama-2 7B, Context Length=40962024.06 | 6.75 | |
| QuIP#Bits=4.0, Model=Llama-2 7B, Context Length=40962024.06 | 6.75 | |
| FP16Model Family=LLaMA, Model Scale=13B, Precision=FP162024.02 | 6.798 | |
| INT4Bits Per FPN=4.00-4.01, Backbone=LLaMA-13b2024.05 | 6.82 | |
| QuIP#Bits=3.0, Model=Llama-1 13B, Context Length=20482024.06 | 6.83 | |
| KVQuant-2b-1%Bits Per FPN=2.32-2.35, Backbone=LLaMA-13b2024.05 | 6.83 | |
| FP16#Bit=FP16, Backbone=LLaMA2-7B2026.04 | 6.84 | |
| QTIPBits=2.0, Model=Llama-2 13B, Context Length=40962024.06 | 6.85 | |
| NF4-gs128Bits Per FPN=4.16, Backbone=LLama-2-7b2024.05 | 6.86 | |
| QTIPBits=3.0, Model=Llama-2 7B, Context Length=40962024.06 | 6.87 | |
| INT4-gs128Bits Per FPN=4.16, Backbone=LLama-2-7b2024.05 | 6.87 | |
| RUQuant#Bit=W4A4, Backbone=LLaMA2-13B2026.04 | 6.87 | |
| RUQuant#Bit=W6A6, Backbone=LLaMA2-7B2026.04 | 6.88 | |
| DuQuant#Bit=W4A4, Backbone=LLaMA2-13B2026.04 | 6.89 | |
| DuQuant#Bit=W6A6, Backbone=LLaMA2-7B2026.04 | 6.89 | |
| aespaModel Family=LLaMA2, Model Scale=13B, Precision=INT42024.02 | 6.891 |