KV Cache Quantization on WikiText-2 prompt 2048-token (train)
0.0026Softmax KL DivergenceBlock-GTQ
Evaluation results
| Method | Links | |
|---|---|---|
| Block-GTQBackbone=DS-R1-Llama-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0026 | |
| Block-GTQBackbone=Mistral-Nemo-12B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0035 | |
| Block-GTQBackbone=Llama-3.1-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0041 | |
| Block-GTQBackbone=Qwen2.5-14B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0056 | |
| TQ-MSEBackbone=DS-R1-Llama-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0063 | |
| Block-GTQBackbone=Qwen3-30B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0088 | |
| TQ-MSEBackbone=Mistral-Nemo-12B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0088 | |
| Block-GTQBackbone=Qwen3-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0091 | |
| TQ-MSEBackbone=Llama-3.1-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0098 | |
| Block-GTQBackbone=DS-R1-Llama-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0099 | |
| Block-GTQBackbone=DS-R1-Qwen-7B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0104 | |
| Block-GTQBackbone=Qwen2.5-3B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0121 | |
| Block-GTQBackbone=Mistral-Nemo-12B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.014 | |
| Block-GTQBackbone=Llama-3.1-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0151 | |
| TQ-MSEBackbone=Qwen2.5-14B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0153 | |
| Block-GTQBackbone=Mean, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0166 | |
| Block-GTQBackbone=Qwen2.5-14B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0214 | |
| TQ-MSEBackbone=DS-R1-Llama-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0245 | |
| Block-GTQBackbone=DS-R1-Qwen-7B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0271 | |
| Block-GTQBackbone=Qwen3-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0327 | |
| TQ-MSEBackbone=Mistral-Nemo-12B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0332 | |
| Block-GTQBackbone=Qwen3-30B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0335 | |
| TQ-MSEBackbone=Llama-3.1-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0367 | |
| Block-GTQBackbone=DS-R1-Llama-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.0378 | |
| KIVIBackbone=Qwen2.5-14B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0405 | |
| KIVIBackbone=DS-V2-Lite, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0435 | |
| Block-GTQBackbone=Qwen2.5-3B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0444 | |
| KIVIBackbone=Qwen3-30B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.049 | |
| KIVIBackbone=Llama-3.1-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.051 | |
| Block-GTQBackbone=Mistral-Nemo-12B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.0515 | |
| Block-GTQBackbone=Mean, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.054 | |
| Block-GTQBackbone=GLM-4-9B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0546 | |
| Block-GTQBackbone=DS-V2-Lite, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0548 | |
| KIVIBackbone=Qwen2.5-14B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0558 | |
| TQ-MSEBackbone=Qwen2.5-14B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0567 | |
| Block-GTQBackbone=Llama-3.1-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.0569 | |
| TQ-MSEBackbone=Qwen3-30B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0571 | |
| KIVIBackbone=DS-R1-Llama-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0584 | |
| TQ-MSEBackbone=Qwen3-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0594 | |
| KIVIBackbone=Llama-3.1-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0614 | |
| KIVIBackbone=DS-R1-Llama-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0663 | |
| KIVIBackbone=Qwen2.5-3B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.068 | |
| KIVIBackbone=DS-V2-Lite, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0684 | |
| KIVIBackbone=Qwen3-30B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0696 | |
| TQ-MSEBackbone=Mean, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0769 | |
| Block-GTQBackbone=Qwen2.5-14B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.0773 | |
| KIVIBackbone=GLM-4-9B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0829 | |
| TQ-MSEBackbone=Qwen2.5-3B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.0859 | |
| KIVIBackbone=Qwen2.5-3B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.0989 | |
| TQ-MSEBackbone=DS-R1-Llama-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1024 | |
| TQ-MSEBackbone=GLM-4-9B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.1072 | |
| KIVIBackbone=GLM-4-9B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.1188 | |
| Block-GTQBackbone=Qwen3-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.121 | |
| Block-GTQBackbone=DS-R1-Qwen-7B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1229 | |
| Block-GTQBackbone=Qwen3-30B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1234 | |
| KIVIBackbone=DS-R1-Llama-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1256 | |
| TQ-MSEBackbone=DS-V2-Lite, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.1274 | |
| TQ-MSEBackbone=Mistral-Nemo-12B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1288 | |
| KIVIBackbone=Llama-3.1-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1298 | |
| TQ-MSEBackbone=Llama-3.1-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1444 | |
| KIVIBackbone=Qwen2.5-14B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1596 | |
| Block-GTQBackbone=GLM-4-9B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.1617 | |
| Block-GTQBackbone=Qwen2.5-3B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.1636 | |
| Block-GTQBackbone=Mean, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.178 | |
| Block-GTQBackbone=DS-V2-Lite, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.1799 | |
| TQ-MSEBackbone=Qwen3-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.196 | |
| KIVIBackbone=Qwen3-30B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.202 | |
| KIVIBackbone=DS-V2-Lite, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.2023 | |
| TQ-MSEBackbone=Mean, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.2097 | |
| TQ-MSEBackbone=Qwen2.5-14B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.2118 | |
| TQ-MSEBackbone=Qwen3-30B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.2288 | |
| TQ-MSEBackbone=Qwen2.5-3B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.2544 | |
| TQ-MSEBackbone=DS-R1-Qwen-7B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.2915 | |
| KIVIBackbone=Mean, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.3053 | |
| KIVIBackbone=Qwen2.5-3B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.3114 | |
| KIVIBackbone=Mean, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.3275 | |
| KIVIBackbone=Qwen3-8B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.3338 | |
| TQ-MSEBackbone=GLM-4-9B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.345 | |
| KIVIBackbone=GLM-4-9B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.3492 | |
| KIVIBackbone=Qwen3-8B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.3594 | |
| TQ-MSEBackbone=DS-V2-Lite, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.4381 | |
| Block-GTQBackbone=GLM-4-9B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.4546 | |
| KIVIBackbone=Mean, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.4708 | |
| TQ-MSEBackbone=DS-R1-Qwen-7B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.484 | |
| KIVIBackbone=Qwen3-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.5126 | |
| KIVIBackbone=DS-R1-Qwen-7B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 0.5326 | |
| KIVIBackbone=DS-R1-Qwen-7B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 0.5569 | |
| Block-GTQBackbone=DS-V2-Lite, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.5709 | |
| TQ-MSEBackbone=Mean, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.5847 | |
| TQ-MSEBackbone=Qwen3-8B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.6349 | |
| TQ-MSEBackbone=Qwen2.5-3B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.6359 | |
| KIVIBackbone=DS-R1-Qwen-7B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.7481 | |
| TQ-MSEBackbone=Qwen3-30B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.8097 | |
| TQ-MSEBackbone=DS-R1-Qwen-7B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.8782 | |
| TQ-MSEBackbone=GLM-4-9B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 0.9397 | |
| TQ-MSEBackbone=DS-V2-Lite, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 1.3615 | |
| KIVIBackbone=Mistral-Nemo-12B, Bit Budget=4 b/dim, Buffer Setting=no-buffer2026.06 | 1.7932 | |
| KIVIBackbone=Mistral-Nemo-12B, Bit Budget=3 b/dim, Buffer Setting=no-buffer2026.06 | 1.8197 | |
| KIVIBackbone=Mistral-Nemo-12B, Bit Budget=2 b/dim, Buffer Setting=no-buffer2026.06 | 1.9675 |