Instruction Following on IFEval (Strict and Loose Accuracy)
81Strict AccuracyFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Model=Qwen3-4B, K/V=16/16, bits/elem=16.00, Uniform Precision=true2026.06 | 81 | 84.3 | |
| KVarNModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 80.4 | 83.4 | |
| KIVIModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 80.3 | 83.2 | |
| HadamardModel=Qwen3-4B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 79.3 | 82.8 | |
| TurboQuantModel=Qwen3-4B, K/V=3/3, bits/elem=4.6, Uniform Precision=true, Implementation=vLLM2026.06 | 79.2 | 81.7 | |
| PolarQuantModel=Qwen3-4B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 79.1 | 82.6 | |
| KittyModel=Qwen3-4B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 78 | 81.9 | |
| KVQuant-1%Model=Qwen3-4B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 76.9 | 80.8 | |
| FP16Model=Llama-3.1-8B, K/V=16/16, bits/elem=16.00, Uniform Precision=true2026.06 | 71.1 | 76.8 | |
| KVarNModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 71 | 76.5 | |
| KIVIModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 70.9 | 74.5 | |
| HadamardModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 70.8 | 75 | |
| KittyModel=Llama-3.1-8B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 70.2 | 75.2 | |
| PolarQuantModel=Llama-3.1-8B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 69.5 | 75 | |
| TurboQuantModel=Llama-3.1-8B, K/V=3/3, bits/elem=4.6, Uniform Precision=true, Implementation=vLLM2026.06 | 66.5 | 71 | |
| FP16Model=Phi-4-14B, K/V=16/16, bits/elem=16.00, Uniform Precision=true2026.06 | 63.6 | 69.3 | |
| KVarNModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 63.4 | 69.2 | |
| KittyModel=Phi-4-14B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 63.2 | 69 | |
| HadamardModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 62.6 | 68.2 | |
| PolarQuantModel=Phi-4-14B, K/V=4/2, bits/elem=3.3, Uniform Precision=true2026.06 | 62.5 | 68.4 | |
| KIVIModel=Phi-4-14B, K/V=2/2, bits/elem=2.3, Uniform Precision=true2026.06 | 60.6 | 67.7 | |
| KVQuant-1%Model=Llama-3.1-8B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 57.7 | 61.4 | |
| TurboQuantModel=Phi-4-14B, K/V=3/3, bits/elem=4.6, Uniform Precision=true, Implementation=vLLM2026.06 | 57.7 | 62.7 | |
| KVQuant-1%Model=Phi-4-14B, K/V=2/2, bits/elem=2.4, Uniform Precision=false2026.06 | 55.1 | 61.2 |