List your company
ResearchFormerly WizWand

Code Generation on HumanEval (Accuracy, Token Count)

88.9AccuracyFP16
74.02877.88981.7585.611Jun 2, 2026
Updated 5mo ago

Evaluation results

MethodLinks
FP16Backbone=Phi-4-14B, K/V=16/16, bits/elem=16.0, UP=✓2026.0688.93,015
FP16Backbone=Qwen3-4B, K/V=16/16, bits/elem=16.0, UP=✓2026.0688.82,764
KVarNBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP=✓2026.0688.43,192
KVarNBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP=✓2026.0688.23,220
TurboQuantBackbone=Phi-4-14B, K/V=3/3, bits/elem=4.6, UP=✓2026.06883,569
QuaRotBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP=✓2026.06873,216
KittyBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.4, UP=×2026.0686.83,085
PolarQuantBackbone=Phi-4-14B, K/V=4/2, bits/elem=3.3, UP=✓2026.0686.83,378
KIVIBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP=✓2026.0686.43,292
QuaRotBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP=✓2026.0686.33,255
TurboQuantBackbone=Qwen3-4B, K/V=3/3, bits/elem=4.6, UP=✓2026.0686.23,269
KVQuant-1%Backbone=Phi-4-14B, K/V=2/2, bits/elem=2.4, UP=×2026.0685.64,017
KVQuant-1%Backbone=Qwen3-4B, K/V=2/2, bits/elem=2.4, UP=×2026.0685.24,594
KittyBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.4, UP=×2026.0682.73,697
PolarQuantBackbone=Qwen3-4B, K/V=4/2, bits/elem=3.3, UP=✓2026.0680.33,811
KIVIBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP=✓2026.0674.64,929