Top-10 Attended-Token Overlap on Attention Diagnostics
94.2Top-10 Attended OverlapBlock-GTQ
Evaluation results
| Method | Links | |
|---|---|---|
| Block-GTQModel=DS-R1-Qwen-7B, Bit Budget=4 b/dim2026.06 | 94.2 | |
| Block-GTQModel=Mistral-Nemo-12B, Bit Budget=4 b/dim2026.06 | 94.2 | |
| Block-GTQModel=DS-R1-Llama-8B, Bit Budget=4 b/dim2026.06 | 93 | |
| Block-GTQModel=Llama-3.1-8B, Bit Budget=4 b/dim2026.06 | 92.9 | |
| Block-GTQModel=Qwen2.5-14B, Bit Budget=4 b/dim2026.06 | 92.6 | |
| Block-GTQModel=Qwen3-8B, Bit Budget=4 b/dim2026.06 | 92.6 | |
| Block-GTQModel=Qwen2.5-3B, Bit Budget=4 b/dim2026.06 | 92.5 | |
| Block-GTQModel=Qwen3-30B, Bit Budget=4 b/dim2026.06 | 92 | |
| TQ-MSEModel=Mistral-Nemo-12B, Bit Budget=4 b/dim2026.06 | 91.2 | |
| Block-GTQModel=DS-R1-Qwen-7B, Bit Budget=3 b/dim2026.06 | 89.7 | |
| TQ-MSEModel=DS-R1-Llama-8B, Bit Budget=4 b/dim2026.06 | 89.6 | |
| TQ-MSEModel=Llama-3.1-8B, Bit Budget=4 b/dim2026.06 | 89.5 | |
| Block-GTQModel=Mistral-Nemo-12B, Bit Budget=3 b/dim2026.06 | 89.1 | |
| TQ-MSEModel=Qwen2.5-14B, Bit Budget=4 b/dim2026.06 | 88.3 | |
| KIVIModel=Llama-3.1-8B, Bit Budget=4 b/dim2026.06 | 87.3 | |
| Block-GTQModel=Llama-3.1-8B, Bit Budget=3 b/dim2026.06 | 87.2 | |
| Block-GTQModel=DS-R1-Llama-8B, Bit Budget=3 b/dim2026.06 | 87.2 | |
| TQ-MSEModel=DS-R1-Qwen-7B, Bit Budget=4 b/dim2026.06 | 87.2 | |
| KIVIModel=DS-R1-Llama-8B, Bit Budget=4 b/dim2026.06 | 87 | |
| TQ-MSEModel=Qwen3-8B, Bit Budget=4 b/dim2026.06 | 86.8 | |
| Block-GTQModel=Qwen3-8B, Bit Budget=3 b/dim2026.06 | 86.7 | |
| TQ-MSEModel=Qwen2.5-3B, Bit Budget=4 b/dim2026.06 | 86.4 | |
| Block-GTQModel=Qwen2.5-14B, Bit Budget=3 b/dim2026.06 | 86.4 | |
| Block-GTQModel=Qwen2.5-3B, Bit Budget=3 b/dim2026.06 | 86.1 | |
| Block-GTQModel=GLM-4-9B, Bit Budget=4 b/dim2026.06 | 86 | |
| KIVIModel=DS-V2-Lite, Bit Budget=4 b/dim2026.06 | 85.9 | |
| KIVIModel=Qwen2.5-3B, Bit Budget=4 b/dim2026.06 | 85.7 | |
| KIVIModel=Qwen2.5-14B, Bit Budget=4 b/dim2026.06 | 85.6 | |
| Block-GTQModel=Qwen3-30B, Bit Budget=3 b/dim2026.06 | 85.5 | |
| KIVIModel=Qwen3-30B, Bit Budget=4 b/dim2026.06 | 85.5 | |
| KIVIModel=DS-R1-Qwen-7B, Bit Budget=4 b/dim2026.06 | 85.1 | |
| KIVIModel=Llama-3.1-8B, Bit Budget=3 b/dim2026.06 | 85 | |
| KIVIModel=DS-R1-Llama-8B, Bit Budget=3 b/dim2026.06 | 84.6 | |
| TQ-MSEModel=Mistral-Nemo-12B, Bit Budget=3 b/dim2026.06 | 84.2 | |
| TQ-MSEModel=Qwen3-30B, Bit Budget=4 b/dim2026.06 | 84.1 | |
| KIVIModel=Qwen3-30B, Bit Budget=3 b/dim2026.06 | 82.5 | |
| KIVIModel=Qwen2.5-14B, Bit Budget=3 b/dim2026.06 | 82.4 | |
| KIVIModel=DS-R1-Qwen-7B, Bit Budget=3 b/dim2026.06 | 82.3 | |
| KIVIModel=Qwen2.5-3B, Bit Budget=3 b/dim2026.06 | 82.1 | |
| Block-GTQModel=DS-V2-Lite, Bit Budget=4 b/dim2026.06 | 81.6 | |
| TQ-MSEModel=DS-R1-Llama-8B, Bit Budget=3 b/dim2026.06 | 81.5 | |
| TQ-MSEModel=Llama-3.1-8B, Bit Budget=3 b/dim2026.06 | 81.4 | |
| Block-GTQModel=DS-R1-Qwen-7B, Bit Budget=2 b/dim2026.06 | 81.4 | |
| KIVIModel=DS-V2-Lite, Bit Budget=3 b/dim2026.06 | 81.2 | |
| KIVIModel=Qwen3-8B, Bit Budget=4 b/dim2026.06 | 81.1 | |
| KIVIModel=GLM-4-9B, Bit Budget=4 b/dim2026.06 | 80.8 | |
| Block-GTQModel=Mistral-Nemo-12B, Bit Budget=2 b/dim2026.06 | 80.5 | |
| TQ-MSEModel=GLM-4-9B, Bit Budget=4 b/dim2026.06 | 79.6 | |
| TQ-MSEModel=DS-R1-Qwen-7B, Bit Budget=3 b/dim2026.06 | 79.4 | |
| TQ-MSEModel=Qwen2.5-14B, Bit Budget=3 b/dim2026.06 | 79.3 | |
| KIVIModel=Mistral-Nemo-12B, Bit Budget=4 b/dim2026.06 | 78.8 | |
| KIVIModel=Qwen3-8B, Bit Budget=3 b/dim2026.06 | 78.5 | |
| TQ-MSEModel=Qwen3-8B, Bit Budget=3 b/dim2026.06 | 77.2 | |
| KIVIModel=Llama-3.1-8B, Bit Budget=2 b/dim2026.06 | 77.2 | |
| Block-GTQModel=Llama-3.1-8B, Bit Budget=2 b/dim2026.06 | 77.1 | |
| Block-GTQModel=DS-R1-Llama-8B, Bit Budget=2 b/dim2026.06 | 77.1 | |
| KIVIModel=Mistral-Nemo-12B, Bit Budget=3 b/dim2026.06 | 76.9 | |
| TQ-MSEModel=Qwen2.5-3B, Bit Budget=3 b/dim2026.06 | 76.8 | |
| Block-GTQModel=Qwen3-8B, Bit Budget=2 b/dim2026.06 | 76.3 | |
| KIVIModel=DS-R1-Llama-8B, Bit Budget=2 b/dim2026.06 | 76.3 | |
| Block-GTQModel=GLM-4-9B, Bit Budget=3 b/dim2026.06 | 76.1 | |
| KIVIModel=GLM-4-9B, Bit Budget=3 b/dim2026.06 | 75.8 | |
| Block-GTQModel=Qwen2.5-14B, Bit Budget=2 b/dim2026.06 | 75.7 | |
| Block-GTQModel=Qwen2.5-3B, Bit Budget=2 b/dim2026.06 | 75.2 | |
| Block-GTQModel=Qwen3-30B, Bit Budget=2 b/dim2026.06 | 74.4 | |
| TQ-MSEModel=DS-V2-Lite, Bit Budget=4 b/dim2026.06 | 73.3 | |
| KIVIModel=DS-R1-Qwen-7B, Bit Budget=2 b/dim2026.06 | 72.9 | |
| KIVIModel=Qwen3-30B, Bit Budget=2 b/dim2026.06 | 72.6 | |
| TQ-MSEModel=Qwen3-30B, Bit Budget=3 b/dim2026.06 | 72.5 | |
| TQ-MSEModel=Mistral-Nemo-12B, Bit Budget=2 b/dim2026.06 | 72.5 | |
| KIVIModel=Qwen2.5-14B, Bit Budget=2 b/dim2026.06 | 72.3 | |
| KIVIModel=Qwen2.5-3B, Bit Budget=2 b/dim2026.06 | 71.6 | |
| KIVIModel=Mistral-Nemo-12B, Bit Budget=2 b/dim2026.06 | 69.5 | |
| KIVIModel=Qwen3-8B, Bit Budget=2 b/dim2026.06 | 69.4 | |
| Block-GTQModel=DS-V2-Lite, Bit Budget=3 b/dim2026.06 | 68.9 | |
| TQ-MSEModel=Llama-3.1-8B, Bit Budget=2 b/dim2026.06 | 68.3 | |
| TQ-MSEModel=DS-R1-Llama-8B, Bit Budget=2 b/dim2026.06 | 68.2 | |
| KIVIModel=DS-V2-Lite, Bit Budget=2 b/dim2026.06 | 68 | |
| TQ-MSEModel=DS-R1-Qwen-7B, Bit Budget=2 b/dim2026.06 | 66.5 | |
| TQ-MSEModel=GLM-4-9B, Bit Budget=3 b/dim2026.06 | 66 | |
| TQ-MSEModel=Qwen2.5-14B, Bit Budget=2 b/dim2026.06 | 64.8 | |
| TQ-MSEModel=Qwen2.5-3B, Bit Budget=2 b/dim2026.06 | 62.1 | |
| KIVIModel=GLM-4-9B, Bit Budget=2 b/dim2026.06 | 61.7 | |
| TQ-MSEModel=Qwen3-8B, Bit Budget=2 b/dim2026.06 | 61.6 | |
| Block-GTQModel=GLM-4-9B, Bit Budget=2 b/dim2026.06 | 60.2 | |
| TQ-MSEModel=DS-V2-Lite, Bit Budget=3 b/dim2026.06 | 55.4 | |
| TQ-MSEModel=Qwen3-30B, Bit Budget=2 b/dim2026.06 | 55.2 | |
| Block-GTQModel=DS-V2-Lite, Bit Budget=2 b/dim2026.06 | 49.9 | |
| TQ-MSEModel=GLM-4-9B, Bit Budget=2 b/dim2026.06 | 47 | |
| TQ-MSEModel=DS-V2-Lite, Bit Budget=2 b/dim2026.06 | 33.9 |