Retrieval and Reasoning on RULER (4K-128K Context Sweep)
98.62Retrieval/Reasoning Score (4K Context)RDKV
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RDKVBackbone=LLaMA-3.1-8B-Instruct, Context budget (B_total)=1024L2026.05 | 98.62 | 98.61 | 95.65 | 88.28 | 80.07 | 66.95 | |
| FullKVBackbone=LLaMA-3.1-8B-Instruct, Context budget (B_total)=1024L2026.05 | 98.58 | 98.88 | 96.98 | 90.33 | 88.49 | 79.91 | |
| AdaKVBackbone=LLaMA-3.1-8B-Instruct, Context budget (B_total)=1024L2026.05 | 92.03 | 85.49 | 80.7 | 75.67 | 72.47 | 58.43 | |
| ThinKBackbone=LLaMA-3.1-8B-Instruct, Context budget (B_total)=1024L2026.05 | 91.76 | 86.44 | 81.45 | 75.94 | 72.44 | 62.85 | |
| SnapKVBackbone=LLaMA-3.1-8B-Instruct, Context budget (B_total)=1024L2026.05 | 89.48 | 84.75 | 80.43 | 75.82 | 72.77 | 62.85 | |
| Snap+ZipBackbone=LLaMA-3.1-8B-Instruct, Context budget (B_total)=1024L2026.05 | 76.49 | 77.84 | 76.26 | 75.05 | 74.59 | 62.37 |