# Long-Context Retrieval on RULER 4K-32K context

Source: [Long-Context Retrieval on RULER 4K-32K context](<https://www.sota2.com/research/sota/long-context-retrieval-on-ruler-4k-32k-context>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:32:08.024Z

Evaluation of model performance on the RULER dataset, specifically measuring retrieval accuracy across context window sizes ranging from 4,000 to 32,000 tokens.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Long-context retrieval |
| Dataset (including any stated split) | RULER 4K-32K context |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/long-context-retrieval-on-ruler-4k-32k-context.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| ManifoldKV | 95.73 | Not recorded | 698abbd9c97a0b3e3b5de23d |
| KeyDiff | 95.66 | Not recorded | 698abbd9c97a0b3e3b5de23d |
| DuoAttention | 95.36 | Not recorded | 698abbd9c97a0b3e3b5de23d |
| KeyDiff | 92.93 | Not recorded | 698abbd9c97a0b3e3b5de23d |
| SnapKV | 83.97 | Not recorded | 698abbd9c97a0b3e3b5de23d |
| SnapKV | 83.97 | Not recorded | 698abbd9c97a0b3e3b5de23d |
| CriticalKV (Value-aware) | 78.9 | Not recorded | 698abbd9c97a0b3e3b5de23d |
| StreamingLLM | 59.3 | Not recorded | 698abbd9c97a0b3e3b5de23d |

### ManifoldKV

Accuracy: 95.73. Result ID: 6998daf95698665ec43f01ca.

[Reporting paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

Recorded evaluation configuration: {"Framework":"AdaKV","Comp.":"0.20","Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### KeyDiff

Accuracy: 95.66. Result ID: 6998daf95698665ec43f01cb.

[Reporting paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a81e209131cea79ade.md>)

Recorded evaluation configuration: {"Framework":"AdaKV","Comp.":"0.20","Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### DuoAttention

Accuracy: 95.36. Result ID: 6998daf95698665ec43f01cd.

[Reporting paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75311e209131cea664b6.md>)

Recorded evaluation configuration: {"Framework":"Architectural/Calibrated","Comp.":"0.20","Model":"Llama-3.1-8B-Instruct","Note":"Requires pre-computed calibration patterns"}. This record does not assert independent reproduction.

### KeyDiff

Accuracy: 92.93. Result ID: 6998daf95698665ec43f01ce.

[Reporting paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a81e209131cea79ade.md>)

Recorded evaluation configuration: {"Framework":"Standalone","Comp.":"0.20","Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### SnapKV

Accuracy: 83.97. Result ID: 6998daf95698665ec43f01cc.

[Reporting paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b21e209131cea5df89.md>)

[Implementation](<https://github.com/FasterDecoding/SnapKV>)

Recorded evaluation configuration: {"Framework":"AdaKV","Comp.":"0.20","Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### SnapKV

Accuracy: 83.97. Result ID: 6998daf95698665ec43f01cf.

[Reporting paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b21e209131cea5df89.md>)

[Implementation](<https://github.com/FasterDecoding/SnapKV>)

Recorded evaluation configuration: {"Framework":"Standalone","Comp.":"0.20","Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### CriticalKV (Value-aware)

Accuracy: 78.9. Result ID: 6998daf95698665ec43f01d0.

[Reporting paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76e11e209131cea6f9a0.md>)

Recorded evaluation configuration: {"Framework":"Standalone","Comp.":"0.20","Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### StreamingLLM

Accuracy: 59.3. Result ID: 6998daf95698665ec43f01d1.

[Reporting paper](<https://www.sota2.com/research/paper/698abbd9c97a0b3e3b5de23d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"Framework":"Standalone","Comp.":"0.20","Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.