Language Modeling on WikiText-2 (test) (Llama-3.1-8B-Instruct)
7.2PerplexityBF16 Baseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BF16 BaselineAvg. Bitwidth=16.0002025.07 | 7.2 | |
| HPTQAvg. Bitwidth=4.1252025.07 | 7.37 | |
| SSQR-5%Avg. Bitwidth=5.7252025.07 | 7.38 | |
| SSQR-4%Avg. Bitwidth=5.4052025.07 | 7.39 | |
| SSQR-3%Avg. Bitwidth=5.0852025.07 | 7.41 | |
| SSQR-2%Avg. Bitwidth=4.7652025.07 | 7.48 | |
| SSQR-1%Avg. Bitwidth=4.4452025.07 | 7.5 | |
| GPTQAvg. Bitwidth=4.1252025.07 | 7.56 | |
| HPTQAvg. Bitwidth=3.1252025.07 | 7.84 | |
| SSQR-5%Avg. Bitwidth=4.7252025.07 | 7.98 | |
| SSQR-4%Avg. Bitwidth=4.4052025.07 | 8.01 | |
| SSQR-3%Avg. Bitwidth=4.0852025.07 | 8.16 | |
| SSQR-2%Avg. Bitwidth=3.7652025.07 | 8.32 | |
| SSQR-1%Avg. Bitwidth=3.4452025.07 | 8.67 | |
| GPTQAvg. Bitwidth=3.1252025.07 | 9.44 | |
| HPTQAvg. Bitwidth=2.1252025.07 | 11.89 | |
| SSQR-5%Avg. Bitwidth=3.7252025.07 | 12.12 | |
| SSQR-4%Avg. Bitwidth=3.4052025.07 | 13.22 | |
| SSQR-3%Avg. Bitwidth=3.0852025.07 | 17.27 | |
| SSQR-2%Avg. Bitwidth=2.7652025.07 | 25.18 | |
| SSQR-1%Avg. Bitwidth=2.4452025.07 | 57.26 | |
| GPTQAvg. Bitwidth=2.1252025.07 | 148.15 |