Long-context Retrieval on Needle-in-a-Haystack 64K-token context
98.7Success RateFull KV Cache
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Full KV CacheContext Length=64K2026.07 | 98.7 | — | |
| DepthWeave-KVContext Length=64K2026.07 | 96.1 | — | |
| TailorKVContext Length=64K2026.07 | 92.6 | — | |
| ChunkKVContext Length=64K2026.07 | 91.2 | — | |
| Eigen AttentionContext Length=64K2026.07 | 90.1 | — | |
| MiniCacheContext Length=64K2026.07 | 89.4 | — | |
| KVSharerContext Length=64K2026.07 | 88.7 | — | |
| PyramidKVContext Length=64K2026.07 | 88.1 | — | |
| SnapKVContext Length=64K2026.07 | 86.5 | — | |
| H2OContext Length=64K2026.07 | 83.8 | — | |
| StreamingLLMContext Length=64K2026.07 | 79.6 | — | |
| Qwen3-30B-A3B-Instruct-2507-FP8 with HGA group-level routingNeedle depth=Overall, fine-tuning=no2026.06 | 3 | — | |
| Qwen3-30B-A3B-Instruct-2507-FP8 with HGA group-level routingNeedle depth=25%, fine-tuning=no2026.06 | — | 948 | |
| Qwen3-30B-A3B-Instruct-2507-FP8 with HGA group-level routingNeedle depth=50%, fine-tuning=no2026.06 | — | 1,053 | |
| Qwen3-30B-A3B-Instruct-2507-FP8 with HGA group-level routingNeedle depth=75%, fine-tuning=no2026.06 | — | 1,332 |