Line Retrieval 200 lines
100AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Model=Phi-4-14B, K/V=16/16, bits/elem=16.0, Uniform Precision=true2026.06 | 100 | |
| FP16Model=Qwen3-4B, K/V=16/16, bits/elem=16.0, Uniform Precision=true2026.06 | 99 | |
| TurboQuantModel=Qwen3-4B, K/V=3/3, bits/elem=4.6, Uniform Precision=true, Implementation=vLLM2026.06 | 99 | |
| KittyModel=Qwen3-4B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 99 | |
| KVarNModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 99 | |
| FP16Model=Llama-3.1-8B, K/V=16/16, bits/elem=16.0, Uniform Precision=true2026.06 | 99 | |
| HadamardModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 99 | |
| PolarQuantModel=Phi-4-14B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 99 | |
| KVarNModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 99 | |
| HadamardModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 98 | |
| KittyModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 98 | |
| KVarNModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 98 | |
| KittyModel=Phi-4-14B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 98 | |
| KIVIModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 97 | |
| KIVIModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 97 | |
| HadamardModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 97 | |
| KIVIModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 96 | |
| TurboQuantModel=Llama-3.1-8B, K/V=3/3, bits/elem=4.8, Uniform Precision=true, Implementation=vLLM2026.06 | 95 | |
| KVQuant-1%Model=Llama-3.1-8B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 88 | |
| PolarQuantModel=Qwen3-4B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 86 | |
| KVQuant-1%Model=Qwen3-4B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 80 | |
| KVQuant-1%Model=Phi-4-14B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 80 | |
| PolarQuantModel=Llama-3.1-8B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 72 | |
| TurboQuantModel=Phi-4-14B, K/V=3/3, bits/elem=4.5, Uniform Precision=true, Implementation=vLLM2026.06 | 51 |