Long-context Retrieval Accuracy on RULER (test)
97.6Accuracy (4K)Full Attention
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Full AttentionModel=Qwen3-8B, Density=100%, Eviction Protocol=Prefill-only eviction2026.06 | 97.6 | 95.43 | 88.46 | 92.79 | — | 93.57 | |
| Full AttentionModel=Qwen3-8B, Density=100%, Eviction Protocol=Prefill + decode eviction2026.06 | 97.6 | 95.43 | 88.46 | 92.79 | — | 93.57 | |
| Full AttentionModel=Llama-3.1-8B, Density=100%, Eviction Protocol=Prefill-only eviction2026.06 | 96.15 | 95.91 | 95.43 | 91.35 | 86.3 | 93.03 | |
| Full AttentionModel=Llama-3.1-8B, Density=100%, Eviction Protocol=Prefill + decode eviction2026.06 | 96.15 | 95.91 | 95.43 | 91.35 | 86.3 | 93.03 | |
| Nexus SamplingModel=Llama-3.1-8B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 91.35 | 91.35 | 94.47 | 89.18 | 84.86 | 90.24 | |
| Nexus SamplingModel=Llama-3.1-8B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 89.42 | 88.46 | 94.23 | 88.46 | 89.18 | 89.95 | |
| MorphKVModel=Llama-3.1-8B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 84.62 | 65.38 | 66.83 | 61.3 | 68.27 | 69.28 | |
| SnapKVModel=Llama-3.1-8B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 82.69 | 68.51 | 73.8 | 64.42 | 73.32 | 72.55 | |
| Nexus SamplingModel=Qwen3-8B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 77.16 | 83.41 | 80.29 | 82.45 | — | 80.83 | |
| Full AttentionModel=Llama-3.2-1B, Density=100%, Eviction Protocol=Prefill-only eviction2026.06 | 76.68 | 70.67 | 65.38 | 64.66 | 60.34 | 67.55 | |
| Full AttentionModel=Llama-3.2-1B, Density=100%, Eviction Protocol=Prefill + decode eviction2026.06 | 76.68 | 70.67 | 65.38 | 64.66 | 60.34 | 67.55 | |
| PyramidKVModel=Llama-3.1-8B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 75 | 83.65 | 86.3 | 86.3 | 81.97 | 82.64 | |
| Nexus SamplingModel=Qwen3-8B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 73.56 | 81.49 | 77.16 | 84.14 | — | 79.09 | |
| SnapKVModel=Qwen3-8B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 70.49 | 67.01 | 73.61 | 79.17 | — | 72.57 | |
| H2OModel=Llama-3.1-8B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 68.03 | 68.99 | 77.4 | 74.52 | 69.23 | 71.63 | |
| MorphKVModel=Qwen3-8B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 67.31 | 62.5 | 61.3 | 57.45 | — | 62.14 | |
| H2OModel=Qwen3-8B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 66.83 | 71.88 | 74.04 | 70.19 | — | 70.74 | |
| PyramidKVModel=Qwen3-8B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 61.54 | 61.06 | 65.14 | 65.38 | — | 63.28 | |
| Nexus SamplingModel=Llama-3.2-1B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 58.89 | 61.78 | 56.97 | 63.7 | 61.54 | 60.58 | |
| Nexus SamplingModel=Llama-3.2-1B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 58.89 | 59.13 | 59.14 | 62.98 | 58.89 | 59.81 | |
| SnapKVModel=Llama-3.2-1B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 45.43 | 42.55 | 37.02 | 37.02 | 30.05 | 38.41 | |
| PyramidKVModel=Llama-3.2-1B, Density=20%, Eviction Protocol=Prefill-only eviction2026.06 | 41.11 | 45.91 | 43.51 | 46.88 | 41.59 | 43.8 | |
| H2OModel=Llama-3.2-1B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 36.06 | 33.65 | 35.34 | 40.15 | 37.26 | 36.49 | |
| MorphKVModel=Llama-3.2-1B, Density=20%, Eviction Protocol=Prefill + decode eviction2026.06 | 22.84 | 17.31 | 15.39 | 18.75 | 12.98 | 17.45 |