Language Modeling on WikiText-103 20w x 2048
9.603Perplexity (PPL)fp16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| fp16bits=16.002026.05 | 9.603 | 0 | |
| HQMQbits=4.99, config=s96 r6 + Med3×2026.05 | 9.621 | 0.019 | |
| HQMQbits=4.51, config=s24 r6 + Med3×2026.05 | 9.701 | 0.098 | |
| HQMQbits=3.79, config=s96 r42026.05 | 10.698 | 1.095 | |
| HQMQbits=3.54, config=s48 r42026.05 | 11.427 | 1.824 | |
| HQMQbits=3.04, config=s24 r32026.05 | 44.564 | 34.96 | |
| naive int4 + Med3×bits=4.622026.05 | 118.977 | 109.4 | |
| naive int4bits=4.002026.05 | 121.715 | 112.1 | |
| naive int3bits=3.002026.05 | 588.222 | 578.6 | |
| naive int2bits=2.002026.05 | 1,529.4 | 1,520 |