KV Cache Quantization on LLaMA-7B
0.01Delta Perplexity (ΔPPL)KVQuant-4b-1%
Evaluation Results
| Method | Links | |
|---|---|---|
| KVQuant-4b-1%Total bits=4.32, Calibration=Yes2026.03 | 0.01 |
Loading the SOTA2 catalog…
| Method | Links | |
|---|---|---|
| KVQuant-4b-1%Total bits=4.32, Calibration=Yes2026.03 | 0.01 |