Long-context Retrieval on NIAH
100AccuracyVanilla
Evaluation Results
| Method | Links | |
|---|---|---|
| VanillaBackbone=Llama 3.3 70B Instruct, Compression Ratio=None, GPU Parallelism=4 GPUs (pipeline parallel)2025.11 | 100 | |
| kvtc 8xBackbone=Llama 3.3 70B Instruct, Compression Ratio=8x, GPU Parallelism=4 GPUs (pipeline parallel)2025.11 | 100 | |
| kvtc 10xBackbone=Llama 3.3 70B Instruct, Compression Ratio=10x, GPU Parallelism=4 GPUs (pipeline parallel)2025.11 | 100 | |
| kvtc 16xBackbone=Llama 3.3 70B Instruct, Compression Ratio=16x, GPU Parallelism=4 GPUs (pipeline parallel)2025.11 | 100 | |
| kvtc 20xBackbone=Llama 3.3 70B Instruct, Compression Ratio=20x, GPU Parallelism=4 GPUs (pipeline parallel)2025.11 | 100 | |
| Full KVBackbone=Qwen2.5-14B, KV Cache Budget=Full2026.06 | 95.28 | |
| Full KVBackbone=Llama-3.1-8B, KV Cache Budget=Full2026.06 | 91.52 | |
| SEKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 91.17 | |
| SEKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 87.42 | |
| Full KVBackbone=Llama-3-8B, KV Cache Budget=Full2026.06 | 85.23 | |
| SentenceKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 84.83 | |
| SemantiCacheBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 83.17 | |
| SentenceKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 82.13 | |
| ChunkKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 81.72 | |
| SEKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 81.34 | |
| SemantiCacheBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 80.74 | |
| ChunkKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 79.23 | |
| PyramidKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 78.41 | |
| SnapKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 76.83 | |
| Full KVBackbone=Llama-3.2-3B, KV Cache Budget=Full2026.06 | 76.72 | |
| PyramidKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 75.83 | |
| SentenceKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 75.47 | |
| SemantiCacheBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 73.83 | |
| SnapKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 73.42 | |
| Full KVBackbone=Mistral-7B, KV Cache Budget=Full2026.06 | 72.39 | |
| ChunkKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 72.14 | |
| SEKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 68.83 | |
| PyramidKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 68.72 | |
| H2OBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 68.47 | |
| SnapKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 66.83 | |
| SEKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 65.74 | |
| H2OBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 64.83 | |
| SentenceKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 63.42 | |
| SemantiCacheBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 61.17 | |
| SentenceKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 60.23 | |
| ChunkKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 59.84 | |
| H2OBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 58.41 | |
| StreamingLLMBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 58.34 | |
| SemantiCacheBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 58.14 | |
| ChunkKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 56.83 | |
| PyramidKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 56.43 | |
| SnapKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 54.17 | |
| StreamingLLMBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 54.17 | |
| PyramidKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 53.17 | |
| SnapKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 51.34 | |
| H2OBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 47.83 | |
| StreamingLLMBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 47.82 | |
| H2OBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 43.72 | |
| StreamingLLMBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 38.43 | |
| StreamingLLMBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 34.71 |