Line Retrieval on Line Retrieval 600 lines
95AccuracyFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Model=Phi-4-14B, K/V=16/16, bits/elem=16.0, Uniform Precision=true2026.06 | 95 | |
| KVarNModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 95 | |
| HadamardModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 94 | |
| FP16Model=Llama-3.1-8B, K/V=16/16, bits/elem=16.0, Uniform Precision=true2026.06 | 92 | |
| FP16Model=Qwen3-4B, K/V=16/16, bits/elem=16.0, Uniform Precision=true2026.06 | 90 | |
| KVarNModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 89 | |
| KittyModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 87 | |
| TurboQuantModel=Qwen3-4B, K/V=3/3, bits/elem=4.6, Uniform Precision=true, Implementation=vLLM2026.06 | 85 | |
| KVarNModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 85 | |
| HadamardModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 85 | |
| PolarQuantModel=Phi-4-14B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 85 | |
| PolarQuantModel=Qwen3-4B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 84 | |
| HadamardModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 83 | |
| KIVIModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 83 | |
| TurboQuantModel=Llama-3.1-8B, K/V=3/3, bits/elem=4.8, Uniform Precision=true, Implementation=vLLM2026.06 | 83 | |
| KittyModel=Phi-4-14B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 83 | |
| KIVIModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 82 | |
| KittyModel=Qwen3-4B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 79 | |
| KIVIModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 74 | |
| KVQuant-1%Model=Phi-4-14B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 67 | |
| PolarQuantModel=Llama-3.1-8B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 58 | |
| KVQuant-1%Model=Qwen3-4B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 57 | |
| TurboQuantModel=Phi-4-14B, K/V=3/3, bits/elem=4.5, Uniform Precision=true, Implementation=vLLM2026.06 | 56 | |
| KVQuant-1%Model=Llama-3.1-8B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 54 |