Language Modeling on WikiText-2 Llama-3.1-8B-Instruct
3.8Delta PPL (%)HyperQuant
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HyperQuantSetting=Weights, rate=4 bps2026.06 | 3.8 | — | — | |
| HIGGSSetting=Weights, rate=4 bps2026.06 | 6.4 | — | — | |
| HyperQuantSetting=Weights, rate=3 bps2026.06 | 22.1 | — | — | |
| HyperQuantSetting=KV cache, rate=1.7 bps2026.06 | 26.9 | — | — | |
| HIGGSSetting=Weights, rate=3 bps2026.06 | 33 | — | — | |
| BF16Setting=Weights+KV cache, int8 MMA, rate=4 bps2026.06 | — | 7.16 | — | |
| HyperQuantSetting=Weights+KV cache, int8 MMA, rate=4 bps2026.06 | — | 7.5 | — | |
| HyperQuantSetting=KV cache, rate=2 bps2026.06 | — | — | 6.4 | |
| TurboQuantSetting=KV cache, rate=2 bps2026.06 | — | — | 3 |