Mathematical Reasoning on UMath
56AccuracyFull Attention
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Full AttentionSelection Method=Full Attn, Budget=None2026.06 | 56 | 9,169 | |
| SnapKV + HARD-KVSelection Method=SnapKV, Config=+ Ours, Budget=40962026.06 | 56 | 21,778 | |
| Full AttentionBackbone=Qwen3-4B, Constraint=Full (No Budget)2026.06 | 53.3 | 10,111.5 | |
| Vanilla + HARD-KVSelection Method=Vanilla, Config=+ Ours, Budget=40962026.06 | 52 | 14,711 | |
| RKV + HARD-KVSelection Method=RKV, Config=+ Ours, Budget=40962026.06 | 52 | 10,123 | |
| HARD-KV (RKV)Budget Size=2048, Variant=+ Ours2026.06 | 52 | 24,748.5 | |
| RKVBackbone=Qwen3-4B, Selection Method=RKV, Top-k Budget=40962026.06 | 52 | 12,825.3 | |
| Full AttentionBudget Size=Full2026.06 | 50.4 | 9,815.7 | |
| HARD-KV (RKV)Budget Size=4096, Variant=+ Ours2026.06 | 50.3 | 21,296.5 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=RKV, Top-k Budget=4096, Top-p Budget=0.902026.06 | 50.3 | 23,156.9 | |
| SnapKVSelection Method=SnapKV, Config=Base, Budget=40962026.06 | 50 | 21,097 | |
| RKVSelection Method=RKV, Config=Base, Budget=40962026.06 | 50 | 10,537 | |
| HARD-KV (SnapKV)Budget Size=4096, Variant=+ Ours2026.06 | 49.4 | 14,520.4 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=SnapKV, Top-k Budget=4096, Top-p Budget=0.902026.06 | 49.4 | 16,209.9 | |
| VanillaSelection Method=Vanilla, Config=Base, Budget=40962026.06 | 48 | 22,183 | |
| SnapKVBudget Size=4096, Variant=Base2026.06 | 46.7 | 12,420.2 | |
| HARD-KV (SnapKV)Budget Size=2048, Variant=+ Ours2026.06 | 46.2 | 22,499.4 | |
| HARD-KV (Vanilla)Budget Size=1024, Variant=+ Ours2026.06 | 46.1 | 15,363.5 | |
| SnapKVBackbone=Qwen3-4B, Selection Method=SnapKV, Top-k Budget=40962026.06 | 44.2 | 14,371.1 | |
| VanillaBackbone=Qwen3-4B, Selection Method=Vanilla, Top-k Budget=40962026.06 | 44 | 12,828 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=SnapKV, Top-k Budget=2048, Top-p Budget=0.902026.06 | 42 | 23,663.2 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=RKV, Top-k Budget=2048, Top-p Budget=0.902026.06 | 42 | 23,663.2 | |
| SnapKVBudget Size=2048, Variant=Base2026.06 | 41.8 | 19,021.4 | |
| RKVBudget Size=4096, Variant=Base2026.06 | 39.9 | 27,429.7 | |
| VanillaBudget Size=2048, Variant=Base2026.06 | 38 | 16,102.5 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=Vanilla, Top-k Budget=4096, Top-p Budget=0.902026.06 | 38 | 24,832.7 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=Vanilla, Top-k Budget=1024, Top-p Budget=0.902026.06 | 36.7 | 23,695.8 | |
| VanillaBackbone=Qwen3-4B, Selection Method=Vanilla, Top-k Budget=20482026.06 | 36.7 | 19,021.4 | |
| HARD-KV (Vanilla)Budget Size=2048, Variant=+ Ours2026.06 | 36.2 | 16,791.5 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=Vanilla, Top-k Budget=2048, Top-p Budget=0.902026.06 | 36.2 | 24,691.8 | |
| HARD-KV (Vanilla)Budget Size=4096, Variant=+ Ours2026.06 | 36.1 | 16,118.8 | |
| RKVBudget Size=2048, Variant=Base2026.06 | 35.1 | 17,915.2 | |
| RKVBackbone=Qwen3-4B, Selection Method=RKV, Top-k Budget=20482026.06 | 35.1 | 17,685.8 | |
| VanillaBudget Size=4096, Variant=Base2026.06 | 32.1 | 26,314.3 | |
| SnapKVBackbone=Qwen3-4B, Selection Method=SnapKV, Top-k Budget=20482026.06 | 31.8 | 20,651.4 | |
| RKVBackbone=Qwen3-4B, Selection Method=RKV, Top-k Budget=10242026.06 | 30.1 | 18,504.6 | |
| RKVBudget Size=1024, Variant=Base2026.06 | 28.1 | 19,625.3 | |
| HARD-KV (RKV)Budget Size=1024, Variant=+ Ours2026.06 | 24.1 | 24,346.1 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=SnapKV, Top-k Budget=1024, Top-p Budget=0.902026.06 | 22.5 | 29,204.8 | |
| HARD-KVBackbone=Qwen3-4B, Selection Method=RKV, Top-k Budget=1024, Top-p Budget=0.902026.06 | 20.1 | 31,351.6 | |
| SnapKVBudget Size=1024, Variant=Base2026.06 | 18 | 22,350.7 | |
| SnapKVBackbone=Qwen3-4B, Selection Method=SnapKV, Top-k Budget=10242026.06 | 18 | 29,332.5 | |
| HARD-KV (SnapKV)Budget Size=1024, Variant=+ Ours2026.06 | 15.5 | 27,705.7 | |
| VanillaBackbone=Qwen3-4B, Selection Method=Vanilla, Top-k Budget=10242026.06 | 14 | 28,859.6 | |
| VanillaBudget Size=1024, Variant=Base2026.06 | 9.8 | 32,768 |