Language Modeling on PTB (test)
8.159PerplexityBaseline
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BaselinePrecision=FP16, Model Family=LLaMA, Model Size=30B2024.02 | 8.159 | — | — | |
| BaselinePrecision=FP16, Model Family=LLaMA, Model Size=30B2024.02 | 8.159 | — | — | |
| aespaPrecision=INT4, Model Family=LLaMA, Model Size=30B2024.02 | 8.283 | — | — | |
| aespaPrecision=INT4, Model Family=LLaMA, Model Size=30B2024.02 | 8.283 | — | — | |
| OmniQuantPrecision=INT4, Model Family=LLaMA, Model Size=30B2024.02 | 8.354 | — | — | |
| OmniQuantPrecision=INT4, Model Family=LLaMA, Model Size=30B2024.02 | 8.354 | — | — | |
| AffineQuantPrecision=INT4, Model Family=LLaMA, Model Size=30B2024.02 | 8.355 | — | — | |
| AffineQuantPrecision=INT4, Model Family=LLaMA, Model Size=30B2024.02 | 8.355 | — | — | |
| aespaPrecision=INT3, Model Family=LLaMA, Model Size=30B2024.02 | 8.684 | — | — | |
| aespaPrecision=INT3, Model Family=LLaMA, Model Size=30B2024.02 | 8.684 | — | — | |
| FP16 baselineModel=OPT-6.7B, Precision=FP162023.08 | 8.76 | — | — | |
| AffineQuantPrecision=INT3, Model Family=LLaMA, Model Size=30B2024.02 | 8.998 | — | — | |
| AffineQuantPrecision=INT3, Model Family=LLaMA, Model Size=30B2024.02 | 8.998 | — | — | |
| OmniQuantPrecision=INT3, Model Family=LLaMA, Model Size=30B2024.02 | 9.065 | — | — | |
| OmniQuantPrecision=INT3, Model Family=LLaMA, Model Size=30B2024.02 | 9.065 | — | — | |
| BaselinePrecision=FP16, Model Family=LLaMA, Model Size=13B2024.02 | 9.081 | — | — | |
| BaselinePrecision=FP16, Model Family=LLaMA, Model Size=13B2024.02 | 9.081 | — | — | |
| aespaPrecision=INT4, Model Family=LLaMA, Model Size=13B2024.02 | 9.277 | — | — | |
| aespaPrecision=INT4, Model Family=LLaMA, Model Size=13B2024.02 | 9.277 | — | — | |
| AffineQuantPrecision=INT4, Model Family=LLaMA, Model Size=13B2024.02 | 9.325 | — | — | |
| AffineQuantPrecision=INT4, Model Family=LLaMA, Model Size=13B2024.02 | 9.325 | — | — | |
| OmniQuantPrecision=INT4, Model Family=LLaMA, Model Size=13B2024.02 | 9.33 | — | — | |
| OmniQuantPrecision=INT4, Model Family=LLaMA, Model Size=13B2024.02 | 9.33 | — | — | |
| aespaPrecision=INT3, Model Family=LLaMA, Model Size=13B2024.02 | 9.818 | — | — | |
| aespaPrecision=INT3, Model Family=LLaMA, Model Size=13B2024.02 | 9.818 | — | — | |
| BaselinePrecision=FP16, Model Family=LLaMA, Model Size=7B2024.02 | 10.12 | — | — | |
| BaselinePrecision=FP16, Model Family=LLaMA, Model Size=7B2024.02 | 10.12 | — | — | |
| OmniQuantPrecision=INT3, Model Family=LLaMA, Model Size=13B2024.02 | 10.24 | — | — | |
| AffineQuantPrecision=INT3, Model Family=LLaMA, Model Size=13B2024.02 | 10.24 | — | — | |
| OmniQuantPrecision=INT3, Model Family=LLaMA, Model Size=13B2024.02 | 10.24 | — | — | |
| AffineQuantPrecision=INT3, Model Family=LLaMA, Model Size=13B2024.02 | 10.24 | — | — | |
| aespaPrecision=INT4, Model Family=LLaMA, Model Size=7B2024.02 | 10.43 | — | — | |
| aespaPrecision=INT4, Model Family=LLaMA, Model Size=7B2024.02 | 10.43 | — | — | |
| AffineQuantPrecision=INT4, Model Family=LLaMA, Model Size=7B2024.02 | 10.53 | — | — | |
| AffineQuantPrecision=INT4, Model Family=LLaMA, Model Size=7B2024.02 | 10.53 | — | — | |
| OmniQuantPrecision=INT4, Model Family=LLaMA, Model Size=7B2024.02 | 10.57 | — | — | |
| OmniQuantPrecision=INT4, Model Family=LLaMA, Model Size=7B2024.02 | 10.57 | — | — | |
| aespaPrecision=INT3, Model Family=LLaMA, Model Size=7B2024.02 | 11.45 | — | — | |
| aespaPrecision=INT3, Model Family=LLaMA, Model Size=7B2024.02 | 11.45 | — | — | |
| TSLDModel=OPT-6.7B, Quantization=Channel-wise2023.08 | 11.6 | — | — | |
| AffineQuantPrecision=INT3, Model Family=LLaMA, Model Size=7B2024.02 | 11.92 | — | — | |
| AffineQuantPrecision=INT3, Model Family=LLaMA, Model Size=7B2024.02 | 11.92 | — | — | |
| OmniQuantPrecision=INT3, Model Family=LLaMA, Model Size=7B2024.02 | 11.98 | — | — | |
| OmniQuantPrecision=INT3, Model Family=LLaMA, Model Size=7B2024.02 | 11.98 | — | — | |
| DenseModel=OPT-175B, Sparsity=0%2023.01 | 12.01 | — | — | |
| LogitModel=OPT-6.7B, Quantization=Channel-wise2023.08 | 12.22 | — | — | |
| SparseGPTModel=OPT-175B, Sparsity=50%2023.01 | 12.37 | — | — | |
| SparseGPTModel=OPT-175B, Sparsity=4:82023.01 | 12.78 | — | — | |
| aespaPrecision=INT2, Model Family=LLaMA, Model Size=30B2024.02 | 12.98 | — | — | |
| aespaPrecision=INT2, Model Family=LLaMA, Model Size=30B2024.02 | 12.98 | — | — | |
| SparseGPTModel=OPT-175B, Sparsity=2:42023.01 | 13.24 | — | — | |
| DenseModel=OPT-66B, Sparsity=0%2023.01 | 13.36 | — | — | |
| DenseModel=OPT-30B, Sparsity=0%2023.01 | 14.04 | — | — | |
| BaselinePrecision=FP16, Model Size=30B2024.02 | 14.04 | — | — | |
| aespaPrecision=INT4, Model Size=30B2024.02 | 14.09 | — | — | |
| Z-FoldPrecision=INT4, Model Size=30B2024.02 | 14.11 | — | — | |
| SparseGPTModel=OPT-66B, Sparsity=50%2023.01 | 14.15 | — | — | |
| OPTQPrecision=INT4, Model Size=30B2024.02 | 14.21 | — | — | |
| aespaPrecision=INT3, Model Size=30B2024.02 | 14.43 | — | — | |
| DenseModel=OPT-13B, Sparsity=0%2023.01 | 14.52 | — | — | |
| BaselinePrecision=FP16, Model Size=13B2024.02 | 14.52 | — | — | |
| Z-FoldPrecision=INT3, Model Size=30B2024.02 | 14.6 | — | — | |
| Z-FoldPrecision=INT4, Model Size=13B2024.02 | 14.65 | — | — | |
| aespaPrecision=INT4, Model Size=13B2024.02 | 14.65 | — | — | |
| SparseGPTModel=OPT-66B, Sparsity=4:82023.01 | 14.68 | — | — | |
| full-prec.Size=474.9 MB, Compression Ratio=1.0x2022.03 | 14.7 | — | — | |
| OPTQPrecision=INT4, Model Size=13B2024.02 | 14.88 | — | — | |
| QuantGPTBits (W-E-A)=8-8-8, Size=121.4 MB, Compression Ratio=3.9x2022.03 | 14.9 | — | — | |
| SparseGPTModel=OPT-30B, Sparsity=50%2023.01 | 14.98 | — | — | |
| QuantGPTBits (W-E-A)=4-4-8, Size=62.4 MB, Compression Ratio=7.6x2022.03 | 15 | — | — | |
| aespaPrecision=INT3, Model Size=13B2024.02 | 15.06 | — | — | |
| OPTQPrecision=INT3, Model Size=30B2024.02 | 15.08 | — | — | |
| Z-FoldPrecision=INT3, Model Size=13B2024.02 | 15.23 | — | — | |
| SparseGPTModel=OPT-66B, Sparsity=2:42023.01 | 15.41 | — | — | |
| RTNPrecision=INT4, Model Size=30B2024.02 | 15.44 | — | — | |
| aespaPrecision=INT2, Model Family=LLaMA, Model Size=13B2024.02 | 15.65 | — | — | |
| aespaPrecision=INT2, Model Family=LLaMA, Model Size=13B2024.02 | 15.65 | — | — | |
| SparseGPTModel=OPT-30B, Sparsity=4:82023.01 | 15.68 | — | — | |
| DenseModel=OPT-6.7B, Sparsity=0%2023.01 | 15.77 | — | — | |
| BaselinePrecision=FP16, Model Family=OPT, Model Size=6.7B2024.02 | 15.77 | — | — | |
| BaselinePrecision=FP16, Model Size=6.7B2024.02 | 15.77 | — | — | |
| BaselinePrecision=FP16, Model Family=OPT, Model Size=6.7B2024.02 | 15.77 | — | — | |
| aespaPrecision=INT4, Model Family=OPT, Model Size=6.7B2024.02 | 15.84 | — | — | |
| aespaPrecision=INT4, Model Size=6.7B2024.02 | 15.84 | — | — | |
| aespaPrecision=INT4, Model Family=OPT, Model Size=6.7B2024.02 | 15.84 | — | — | |
| SparseGPTModel=OPT-13B, Sparsity=50%2023.01 | 15.97 | — | — | |
| Z-FoldPrecision=INT4, Model Size=6.7B2024.02 | 15.98 | — | — | |
| AffineQuantPrecision=INT4, Model Family=OPT, Model Size=6.7B2024.02 | 16.02 | — | — | |
| AffineQuantPrecision=INT4, Model Family=OPT, Model Size=6.7B2024.02 | 16.02 | — | — | |
| OmniQuantPrecision=INT4, Model Family=OPT, Model Size=6.7B2024.02 | 16.04 | — | — | |
| OmniQuantPrecision=INT4, Model Family=OPT, Model Size=6.7B2024.02 | 16.04 | — | — | |
| QuantGPTBits (W-E-A)=2-2-8, Size=33.0 MB, Compression Ratio=14.4x2022.03 | 16.1 | — | — | |
| aespaPrecision=INT3, Model Family=OPT, Model Size=6.7B2024.02 | 16.28 | — | — | |
| aespaPrecision=INT3, Model Size=6.7B2024.02 | 16.28 | — | — | |
| aespaPrecision=INT3, Model Family=OPT, Model Size=6.7B2024.02 | 16.28 | — | — | |
| RTNPrecision=INT4, Model Size=13B2024.02 | 16.41 | — | — | |
| OPTQPrecision=INT4, Model Size=6.7B2024.02 | 16.42 | — | — | |
| OPTQPrecision=INT3, Model Size=13B2024.02 | 16.42 | — | — | |
| SparseGPTModel=OPT-30B, Sparsity=2:42023.01 | 16.62 | — | — | |
| Z-FoldPrecision=INT3, Model Size=6.7B2024.02 | 16.64 | — | — |