List your company
ResearchFormerly WizWand

Long-context Variable Tracking on variable-tracking

79.5AccuracyQwen2.5-72B Baseline
27.54154.568Apr 23, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Qwen2.5-72B BaselineModel scale=72B, Compression Method=Baseline, KV retention=100%2026.0479.5
Sub-token routing (Qwen2.5-72B)Model scale=72B, Compression Method=Sub-token routing, KV retention=62.5%2026.0479.5
Expected Attention (Qwen2.5-72B)Model scale=72B, Compression Method=EA, KV retention=62.5%2026.0479.5
EA + Sub-token routing (Qwen2.5-72B)Model scale=72B, Compression Method=EA+Sub-token, KV retention=37.5%2026.0479.5
Qwen2.5-32B BaselineModel scale=32B, Compression Method=Baseline, KV retention=100%2026.0479
Sub-token routing (Qwen2.5-32B)Model scale=32B, Compression Method=Sub-token routing, KV retention=62.5%2026.0479
Expected Attention (Qwen2.5-32B)Model scale=32B, Compression Method=EA, KV retention=62.5%2026.0479
EA + Sub-token routing (Qwen2.5-32B)Model scale=32B, Compression Method=EA+Sub-token, KV retention=37.5%2026.0479
Qwen2.5-14B BaselineModel scale=14B, Compression Method=Baseline, KV retention=100%2026.0477
Sub-token routing (Qwen2.5-14B)Model scale=14B, Compression Method=Sub-token routing, KV retention=62.5%2026.0477
Expected Attention (Qwen2.5-14B)Model scale=14B, Compression Method=EA, KV retention=62.5%2026.0477
EA + Sub-token routing (Qwen2.5-14B)Model scale=14B, Compression Method=EA+Sub-token, KV retention=37.5%2026.0477
Qwen2.5-7B BaselineModel scale=7B, Compression Method=Baseline, KV retention=100%2026.0429.5
Sub-token routing (Qwen2.5-7B)Model scale=7B, Compression Method=Sub-token routing, KV retention=62.5%2026.0429.5
Expected Attention (Qwen2.5-7B)Model scale=7B, Compression Method=EA, KV retention=62.5%2026.0429.5
EA + Sub-token routing (Qwen2.5-7B)Model scale=7B, Compression Method=EA+Sub-token, KV retention=37.5%2026.0429.5