List your company
ResearchFormerly WizWand

Needle-in-a-Haystack 4x Original Context

100AccuracyTransactional Attention
-4235077Dec 13, 2025Jan 2, 2026Jan 22, 2026Feb 11, 2026Mar 3, 2026Mar 23, 2026Apr 13, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Transactional AttentionRetention Budget (K)=16, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
Transactional AttentionRetention Budget (K)=32, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
Transactional AttentionRetention Budget (K)=64, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
Transactional AttentionRetention Budget (K)=128, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
Transactional AttentionRetention Budget (K)=256, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
TOVARetention Budget (K)=128, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
TOVARetention Budget (K)=256, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
SnapKVRetention Budget (K)=128, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
SnapKVRetention Budget (K)=256, Base Model=Llama-3.2-1B, Context Length=4K2026.04100
SmolLM-DroPERoPE scaling method=DroPE2025.1255
PyramidKVRetention Budget (K)=256, Base Model=Llama-3.2-1B, Context Length=4K2026.0446
SmolLM + LongRoPE2RoPE scaling method=LongRoPE22025.1226.2
SmolLM + YaRNRoPE scaling method=YaRN2025.1225.62
SmolLM + RoPE-NTKRoPE scaling method=RoPE-NTK2025.1214.37
TOVARetention Budget (K)=16, Base Model=Llama-3.2-1B, Context Length=4K2026.040
TOVARetention Budget (K)=32, Base Model=Llama-3.2-1B, Context Length=4K2026.040
TOVARetention Budget (K)=64, Base Model=Llama-3.2-1B, Context Length=4K2026.040
H2ORetention Budget (K)=16, Base Model=Llama-3.2-1B, Context Length=4K2026.040
H2ORetention Budget (K)=32, Base Model=Llama-3.2-1B, Context Length=4K2026.040
H2ORetention Budget (K)=64, Base Model=Llama-3.2-1B, Context Length=4K2026.040
H2ORetention Budget (K)=128, Base Model=Llama-3.2-1B, Context Length=4K2026.040
H2ORetention Budget (K)=256, Base Model=Llama-3.2-1B, Context Length=4K2026.040
SnapKVRetention Budget (K)=16, Base Model=Llama-3.2-1B, Context Length=4K2026.040
SnapKVRetention Budget (K)=32, Base Model=Llama-3.2-1B, Context Length=4K2026.040
SnapKVRetention Budget (K)=64, Base Model=Llama-3.2-1B, Context Length=4K2026.040
StreamingLLMRetention Budget (K)=16, Base Model=Llama-3.2-1B, Context Length=4K2026.040
StreamingLLMRetention Budget (K)=32, Base Model=Llama-3.2-1B, Context Length=4K2026.040
StreamingLLMRetention Budget (K)=64, Base Model=Llama-3.2-1B, Context Length=4K2026.040
StreamingLLMRetention Budget (K)=128, Base Model=Llama-3.2-1B, Context Length=4K2026.040
StreamingLLMRetention Budget (K)=256, Base Model=Llama-3.2-1B, Context Length=4K2026.040
PyramidKVRetention Budget (K)=16, Base Model=Llama-3.2-1B, Context Length=4K2026.040
PyramidKVRetention Budget (K)=32, Base Model=Llama-3.2-1B, Context Length=4K2026.040
PyramidKVRetention Budget (K)=64, Base Model=Llama-3.2-1B, Context Length=4K2026.040
PyramidKVRetention Budget (K)=128, Base Model=Llama-3.2-1B, Context Length=4K2026.040
DynamicKVRetention Budget (K)=16, Base Model=Llama-3.2-1B, Context Length=4K2026.040