Long-context Retrieval and Reasoning on RULER 128K context
64AccuracyFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Model=Qwen2.5-32B2026.03 | 64 | 2,350 | |
| TTKVModel=Qwen2.5-32B2026.03 | 63.9 | 550 | |
| FP16Model=DeepSeek-R1-14B2026.03 | 63.5 | 2,550 | |
| TTKVModel=DeepSeek-R1-14B2026.03 | 63.3 | 460 | |
| DiffKVModel=Qwen2.5-32B2026.03 | 62.5 | 885 | |
| KVQuantModel=Qwen2.5-32B2026.03 | 62.3 | 965 | |
| ShadowKVModel=Qwen2.5-32B2026.03 | 62.3 | 865 | |
| DiffKVModel=DeepSeek-R1-14B2026.03 | 62.1 | 915 | |
| FP16Model=Llama-3.1-70B2026.03 | 62 | 2,450 | |
| ShadowKVModel=DeepSeek-R1-14B2026.03 | 61.9 | 885 | |
| TTKVModel=Llama-3.1-70B2026.03 | 61.8 | 590 | |
| KVQuantModel=DeepSeek-R1-14B2026.03 | 61.8 | 985 | |
| DiffKVModel=Llama-3.1-70B2026.03 | 61 | 875 | |
| KVQuantModel=Llama-3.1-70B2026.03 | 60.8 | 965 | |
| ShadowKVModel=Llama-3.1-70B2026.03 | 60.8 | 845 | |
| KIVIModel=Qwen2.5-32B2026.03 | 60.5 | 1,015 | |
| KIVIModel=DeepSeek-R1-14B2026.03 | 59.8 | 1,045 | |
| KIVIModel=Llama-3.1-70B2026.03 | 59.5 | 1,015 |