Language Modeling on C4 (Perplexity and Efficiency Metrics)
12.701Perplexityfp16 baseline
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| fp16 baselinebits=–, context length=4096, symmetric K=V=True2026.05 | 12.701 | 0 | 1 | |
| OCTOPUS-QJLbits=4, context length=4096, symmetric K=V=True2026.05 | 12.893 | 1.5 | 2 | |
| OCTOPUSbits=4, context length=4096, symmetric K=V=True2026.05 | 12.896 | 1.5 | 2.2 | |
| TurboQuant-MSEbits=4, context length=4096, symmetric K=V=True2026.05 | 12.921 | 1.7 | 2.2 | |
| PolarQuantbits=4, context length=4096, symmetric K=V=True2026.05 | 13.091 | 3.1 | 2.2 | |
| OCTOPUSbits=3, context length=4096, symmetric K=V=True2026.05 | 13.446 | 5.9 | 2.5 | |
| OCTOPUS-QJLbits=3, context length=4096, symmetric K=V=True2026.05 | 13.474 | 6.1 | 2.3 | |
| TurboQuant-QJLbits=4, context length=4096, symmetric K=V=True2026.05 | 13.699 | 7.9 | 2.2 | |
| TurboQuant-MSEbits=3, context length=4096, symmetric K=V=True2026.05 | 13.761 | 8.3 | 2.6 | |
| PolarQuantbits=3, context length=4096, symmetric K=V=True2026.05 | 14.716 | 15.9 | 2.6 | |
| OCTOPUS-QJLbits=2, context length=4096, symmetric K=V=True2026.05 | 17.955 | 41.4 | 2.6 | |
| OCTOPUSbits=2, context length=4096, symmetric K=V=True2026.05 | 17.976 | 41.5 | 2.9 | |
| TurboQuant-QJLbits=3, context length=4096, symmetric K=V=True2026.05 | 20.308 | 59.9 | 2.5 | |
| TurboQuant-MSEbits=2, context length=4096, symmetric K=V=True2026.05 | 22.536 | 77.4 | 3 | |
| PolarQuantbits=2, context length=4096, symmetric K=V=True2026.05 | 61.486 | 384.1 | 3 | |
| TurboQuant-QJLbits=2, context length=4096, symmetric K=V=True2026.05 | 184.034 | 1,349 | 3 |