Language Modeling on Wiki2 (Delta PPL)
0.46Delta PPL (Llama2-7B)QuaRot
Evaluation results
| Method | Links | ||
|---|---|---|---|
| QuaRotBitwidth (W4A4)=4.13, Group size=g1282025.02 | 0.46 | 0.29 | |
| AtomBitwidth (W4A4)=4.13, Group size=g1282025.02 | 0.56 | 0.36 | |
| OmniQuantBitwidth (W4A4)=4.13, Group size=g1282025.02 | 9.14 | — | |
| SmoothQuantBitwidth (W4A4)=4.13, Group size=g1282025.02 | 77.65 | — |