List your company
ResearchFormerly WizWand

Long-context Retrieval on NIAH

100AccuracyVanilla
32.098449.726767.35584.9833Nov 3, 2025Dec 12, 2025Jan 21, 2026Mar 2, 2026Apr 11, 2026May 21, 2026Jun 30, 2026
Updated 4mo ago

Evaluation results

MethodLinks
VanillaBackbone=Llama 3.3 70B Instruct, Compression Ratio=None, GPU Parallelism=4 GPUs (pipeline parallel)2025.11100
kvtc 8xBackbone=Llama 3.3 70B Instruct, Compression Ratio=8x, GPU Parallelism=4 GPUs (pipeline parallel)2025.11100
kvtc 10xBackbone=Llama 3.3 70B Instruct, Compression Ratio=10x, GPU Parallelism=4 GPUs (pipeline parallel)2025.11100
kvtc 16xBackbone=Llama 3.3 70B Instruct, Compression Ratio=16x, GPU Parallelism=4 GPUs (pipeline parallel)2025.11100
kvtc 20xBackbone=Llama 3.3 70B Instruct, Compression Ratio=20x, GPU Parallelism=4 GPUs (pipeline parallel)2025.11100
Full KVBackbone=Qwen2.5-14B, KV Cache Budget=Full2026.0695.28
Full KVBackbone=Llama-3.1-8B, KV Cache Budget=Full2026.0691.52
SEKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.0691.17
SEKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.0687.42
Full KVBackbone=Llama-3-8B, KV Cache Budget=Full2026.0685.23
SentenceKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.0684.83
SemantiCacheBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.0683.17
SentenceKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.0682.13
ChunkKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.0681.72
SEKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.0681.34
SemantiCacheBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.0680.74
ChunkKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.0679.23
PyramidKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.0678.41
SnapKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.0676.83
Full KVBackbone=Llama-3.2-3B, KV Cache Budget=Full2026.0676.72
PyramidKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.0675.83
SentenceKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.0675.47
SemantiCacheBackbone=Llama-3-8B, KV Cache Budget=10%2026.0673.83
SnapKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.0673.42
Full KVBackbone=Mistral-7B, KV Cache Budget=Full2026.0672.39
ChunkKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.0672.14
SEKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.0668.83
PyramidKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.0668.72
H2OBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.0668.47
SnapKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.0666.83
SEKVBackbone=Mistral-7B, KV Cache Budget=10%2026.0665.74
H2OBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.0664.83
SentenceKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.0663.42
SemantiCacheBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.0661.17
SentenceKVBackbone=Mistral-7B, KV Cache Budget=10%2026.0660.23
ChunkKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.0659.84
H2OBackbone=Llama-3-8B, KV Cache Budget=10%2026.0658.41
StreamingLLMBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.0658.34
SemantiCacheBackbone=Mistral-7B, KV Cache Budget=10%2026.0658.14
ChunkKVBackbone=Mistral-7B, KV Cache Budget=10%2026.0656.83
PyramidKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.0656.43
SnapKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.0654.17
StreamingLLMBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.0654.17
PyramidKVBackbone=Mistral-7B, KV Cache Budget=10%2026.0653.17
SnapKVBackbone=Mistral-7B, KV Cache Budget=10%2026.0651.34
H2OBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.0647.83
StreamingLLMBackbone=Llama-3-8B, KV Cache Budget=10%2026.0647.82
H2OBackbone=Mistral-7B, KV Cache Budget=10%2026.0643.72
StreamingLLMBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.0638.43
StreamingLLMBackbone=Mistral-7B, KV Cache Budget=10%2026.0634.71