Language Modeling on WikiText-103 (Bits and Perplexity Metrics)
2Bits Per Character (BPC)naive int2
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| naive int2measurement context=50w x 20482026.05 | 2 | 1,010.6 | 1,004 | |
| naive int3measurement context=50w x 20482026.05 | 3 | 16.648 | 10.37 | |
| hqmq_s24_r3measurement context=50w x 20482026.05 | 3.04 | 7.023 | 0.745 | |
| sph_r4_jl4 (TurboQuant-style)measurement context=50w x 20482026.05 | 3.15 | 7.396 | 1.118 | |
| hqmq_s48_r4measurement context=50w x 20482026.05 | 3.54 | 6.572 | 0.294 | |
| hqmq_s96_r4measurement context=50w x 20482026.05 | 3.79 | 6.479 | 0.201 | |
| naive int4measurement context=50w x 20482026.05 | 4 | 6.811 | 0.533 | |
| hqmq_s192_r4measurement context=20w x 20482026.05 | 4.04 | 6.387 | 0.142 | |
| hqmq_s24_r6 + Med3xmeasurement context=20w x 20482026.05 | 4.27 | 6.586 | 0.341 | |
| hqmq_s192_r6measurement context=20w x 20482026.05 | 4.54 | 6.333 | 0.088 | |
| hqmq_s96_r6 + Med3xmeasurement context=20w x 20482026.05 | 4.76 | 6.367 | 0.122 | |
| hqmq_s192_r6 + Med3xmeasurement context=20w x 20482026.05 | 5 | 6.317 | 0.071 | |
| fp16measurement context=50w x 20482026.05 | 16 | 6.278 | 0 |