Mathematical Reasoning on AIME 24 (Accuracy and Token Count)
79.9AccuracyFull Attention
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Full AttentionBudget Size=Full2026.06 | 79.9 | 13,735.7 | |
| FP16Backbone=Phi-4-14B, K/V=16/16, bits/elem=16.0, UP (Uniform Precision)=true2026.06 | 62.2 | 11 | |
| KVarNBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 61.7 | 11 | |
| FP16Backbone=Qwen3-4B, K/V=16/16, bits/elem=16.0, UP (Uniform Precision)=true2026.06 | 61.1 | 12 | |
| KVarNBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 60 | 12 | |
| PolarQuantBackbone=Phi-4-14B, K/V=4/2, bits/elem=3.3, UP (Uniform Precision)=true2026.06 | 60 | 12 | |
| QuaRotBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 58.9 | 12 | |
| HARD-KV (RKV)Budget Size=4096, Variant=+ Ours2026.06 | 57.9 | 28,094 | |
| KIVIBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 57.8 | 12 | |
| SnapKVBudget Size=4096, Variant=Base2026.06 | 57.7 | 19,157.4 | |
| QuaRotBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 56.7 | 12 | |
| HARD-KV (SnapKV)Budget Size=4096, Variant=+ Ours2026.06 | 56.7 | 21,616.5 | |
| KVQuant-1%Backbone=Phi-4-14B, K/V=2/2, bits/elem=2.4, UP (Uniform Precision)=false2026.06 | 55.6 | 13 | |
| KittyBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.4, UP (Uniform Precision)=false2026.06 | 55.6 | 12 | |
| KIVIBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 55.5 | 12 | |
| KittyBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.4, UP (Uniform Precision)=false2026.06 | 53.3 | 13 | |
| HARD-KV (RKV)Budget Size=2048, Variant=+ Ours2026.06 | 53.3 | 27,819.6 | |
| PolarQuantBackbone=Qwen3-4B, K/V=4/2, bits/elem=3.3, UP (Uniform Precision)=true2026.06 | 52.2 | 13 | |
| TurboQuantBackbone=Phi-4-14B, K/V=3/3, bits/elem=4.5, UP (Uniform Precision)=true, Implementation=vLLM2026.06 | 52.2 | 13 | |
| TurboQuantBackbone=Qwen3-4B, K/V=3/3, bits/elem=4.6, UP (Uniform Precision)=true, Implementation=vLLM2026.06 | 48.9 | 13 | |
| RKVBudget Size=4096, Variant=Base2026.06 | 43.3 | 23,281.5 | |
| HARD-KV (SnapKV)Budget Size=2048, Variant=+ Ours2026.06 | 40.2 | 28,471.4 | |
| KVQuant-1%Backbone=Qwen3-4B, K/V=2/2, bits/elem=2.4, UP (Uniform Precision)=false2026.06 | 40 | 14 | |
| HARD-KV (Vanilla)Budget Size=2048, Variant=+ Ours2026.06 | 37.3 | 24,158.1 | |
| HARD-KV (Vanilla)Budget Size=4096, Variant=+ Ours2026.06 | 33.5 | 23,449.8 | |
| VanillaBudget Size=2048, Variant=Base2026.06 | 33.1 | 23,530.6 | |
| VanillaBudget Size=4096, Variant=Base2026.06 | 29.7 | 31,843.5 | |
| HARD-KV (Vanilla)Budget Size=1024, Variant=+ Ours2026.06 | 24 | 24,158.1 | |
| HARD-KV (RKV)Budget Size=1024, Variant=+ Ours2026.06 | 23.4 | 30,480.8 | |
| RKVBudget Size=2048, Variant=Base2026.06 | 23.3 | 27,968 | |
| SnapKVBudget Size=2048, Variant=Base2026.06 | 21.3 | 23,860.3 | |
| HARD-KV (SnapKV)Budget Size=1024, Variant=+ Ours2026.06 | 13.3 | 32,768 | |
| RKVBudget Size=1024, Variant=Base2026.06 | 13.3 | 30,423.2 | |
| VanillaBudget Size=1024, Variant=Base2026.06 | 10 | 32,161 | |
| SnapKVBudget Size=1024, Variant=Base2026.06 | 6.2 | 31,323.1 |