Code Generation on HumanEval (Accuracy, Token Count)
88.9AccuracyFP16
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FP16Backbone=Phi-4-14B, K/V=16/16, bits/elem=16.0, UP=✓2026.06 | 88.9 | 3,015 | |
| FP16Backbone=Qwen3-4B, K/V=16/16, bits/elem=16.0, UP=✓2026.06 | 88.8 | 2,764 | |
| KVarNBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP=✓2026.06 | 88.4 | 3,192 | |
| KVarNBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP=✓2026.06 | 88.2 | 3,220 | |
| TurboQuantBackbone=Phi-4-14B, K/V=3/3, bits/elem=4.6, UP=✓2026.06 | 88 | 3,569 | |
| QuaRotBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP=✓2026.06 | 87 | 3,216 | |
| KittyBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.4, UP=×2026.06 | 86.8 | 3,085 | |
| PolarQuantBackbone=Phi-4-14B, K/V=4/2, bits/elem=3.3, UP=✓2026.06 | 86.8 | 3,378 | |
| KIVIBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP=✓2026.06 | 86.4 | 3,292 | |
| QuaRotBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP=✓2026.06 | 86.3 | 3,255 | |
| TurboQuantBackbone=Qwen3-4B, K/V=3/3, bits/elem=4.6, UP=✓2026.06 | 86.2 | 3,269 | |
| KVQuant-1%Backbone=Phi-4-14B, K/V=2/2, bits/elem=2.4, UP=×2026.06 | 85.6 | 4,017 | |
| KVQuant-1%Backbone=Qwen3-4B, K/V=2/2, bits/elem=2.4, UP=×2026.06 | 85.2 | 4,594 | |
| KittyBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.4, UP=×2026.06 | 82.7 | 3,697 | |
| PolarQuantBackbone=Qwen3-4B, K/V=4/2, bits/elem=3.3, UP=✓2026.06 | 80.3 | 3,811 | |
| KIVIBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP=✓2026.06 | 74.6 | 4,929 |