Language Modeling on Llama-2 7B base (PPL, KL)
4.7527PerplexityBF16 (ref)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BF16 (ref)bpw=16.002026.05 | 4.7527 | — | — | |
| SmoothQuant W8A8bpw=8.132026.05 | 4.7576 | 0.1 | 0.0009 | |
| QAM-W-5.5bpw=5.512026.05 | 4.7664 | 0.29 | 0.0043 | |
| QTIP 4-bit (relaxml)bpw=4.002026.05 | 4.8155 | 1.32 | 0.01 | |
| QAM-W-4bpw=4.002026.05 | 4.8551 | 2.15 | 0.0292 | |
| QTIP 2-bit (relaxml)bpw=2.002026.05 | 5.5841 | 17.49 | 0.1195 | |
| AQLM 2-bit 1×16 (ISTA-DASLab)bpw=2.032026.05 | 5.6716 | 19.33 | 0.1337 |