Language Modeling on WikiText-103 50w x 2048 (test)
7.59Perplexityfp16
Evaluation results
| Method | Links | |
|---|---|---|
| fp16bits=16.002026.05 | 7.59 | |
| HQMQ s192 r6 + Med3xbits=5.152026.05 | 8.83 | |
| HQMQ s96 r6 + Med3xbits=4.912026.05 | 9.03 | |
| HQMQ s24 r6 + Med3xbits=4.422026.05 | 9.69 | |
| HQMQ s96 r6bits=4.292026.05 | 98.4 | |
| HQMQ s192 r6bits=4.542026.05 | 109 | |
| HQMQ s24 r6bits=3.792026.05 | 197 | |
| naive int4 + Med3xbits=4.592026.05 | 10,668 | |
| naive int4 + Med5xbits=4.372026.05 | 12,738 | |
| naive int3 + Med3xbits=3.622026.05 | 13,350 | |
| naive int4bits=4.002026.05 | 17,661 | |
| naive int3bits=3.002026.05 | 25,411 |