# Retrieval and Reasoning on RULER (4K-128K Context Sweep)

Source: [Retrieval and Reasoning on RULER (4K-128K Context Sweep)](<https://www.sota2.com/research/sota/retrieval-and-reasoning-on-ruler-4k-128k-context-sweep>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-12T07:52:28.300Z

Evaluation of model performance on the RULER dataset across a range of context lengths (4K, 8K, 16K, 32K, 64K, 128K) for retrieval and reasoning tasks.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Retrieval and reasoning |
| Dataset (including any stated split) | RULER |
| Metric | Retrieval/Reasoning Score (4K Context) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Retrieval/Reasoning Score (4K Context)](<https://www.sota2.com/research/sota/retrieval-and-reasoning-on-ruler-4k-128k-context-sweep.md?metric=Retrieval%2FReasoning%20Score%20(4K%20Context)>): Higher is better; unit: PERCENT.

- [Retrieval/Reasoning Score (8K Context)](<https://www.sota2.com/research/sota/retrieval-and-reasoning-on-ruler-4k-128k-context-sweep.md?metric=Retrieval%2FReasoning%20Score%20(8K%20Context)>): Higher is better; unit: PERCENT.

- [Retrieval/Reasoning Score (16K Context)](<https://www.sota2.com/research/sota/retrieval-and-reasoning-on-ruler-4k-128k-context-sweep.md?metric=Retrieval%2FReasoning%20Score%20(16K%20Context)>): Higher is better; unit: PERCENT.

- [Retrieval/Reasoning Score (32K Context)](<https://www.sota2.com/research/sota/retrieval-and-reasoning-on-ruler-4k-128k-context-sweep.md?metric=Retrieval%2FReasoning%20Score%20(32K%20Context)>): Higher is better; unit: PERCENT.

- [Retrieval/Reasoning Score (64K Context)](<https://www.sota2.com/research/sota/retrieval-and-reasoning-on-ruler-4k-128k-context-sweep.md?metric=Retrieval%2FReasoning%20Score%20(64K%20Context)>): Higher is better; unit: PERCENT.

- [Retrieval/Reasoning Score (128K Context)](<https://www.sota2.com/research/sota/retrieval-and-reasoning-on-ruler-4k-128k-context-sweep.md?metric=Retrieval%2FReasoning%20Score%20(128K%20Context)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Retrieval/Reasoning Score (4K Context) | Result date | Report paper |
| --- | --- | --- | --- |
| RDKV | 98.62 | Not recorded | 6a02b3f7dbc836b87d824593 |
| FullKV | 98.58 | Not recorded | 6a02b3f7dbc836b87d824593 |
| AdaKV | 92.03 | Not recorded | 6a02b3f7dbc836b87d824593 |
| ThinK | 91.76 | Not recorded | 6a02b3f7dbc836b87d824593 |
| SnapKV | 89.48 | Not recorded | 6a02b3f7dbc836b87d824593 |
| Snap+Zip | 76.49 | Not recorded | 6a02b3f7dbc836b87d824593 |

### RDKV

Retrieval/Reasoning Score (4K Context): 98.62. Result ID: 6a02dc2c47f34ce924e56ad8.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b3f7dbc836b87d824593.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b3f7dbc836b87d824593.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B-Instruct","Context budget (B\_total)":"1024L"}. This record does not assert independent reproduction.

### FullKV

Retrieval/Reasoning Score (4K Context): 98.58. Result ID: 6a02dc2c47f34ce924e56ad3.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b3f7dbc836b87d824593.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B-Instruct","Context budget (B\_total)":"1024L"}. This record does not assert independent reproduction.

### AdaKV

Retrieval/Reasoning Score (4K Context): 92.03. Result ID: 6a02dc2c47f34ce924e56ad5.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b3f7dbc836b87d824593.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c2e1e209131cea8ce46.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B-Instruct","Context budget (B\_total)":"1024L"}. This record does not assert independent reproduction.

### ThinK

Retrieval/Reasoning Score (4K Context): 91.76. Result ID: 6a02dc2c47f34ce924e56ad6.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b3f7dbc836b87d824593.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa772c1e209131cea7190c.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B-Instruct","Context budget (B\_total)":"1024L"}. This record does not assert independent reproduction.

### SnapKV

Retrieval/Reasoning Score (4K Context): 89.48. Result ID: 6a02dc2c47f34ce924e56ad4.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b3f7dbc836b87d824593.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b21e209131cea5df89.md>)

[Implementation](<https://github.com/FasterDecoding/SnapKV>)

Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B-Instruct","Context budget (B\_total)":"1024L"}. This record does not assert independent reproduction.

### Snap+Zip

Retrieval/Reasoning Score (4K Context): 76.49. Result ID: 6a02dc2c47f34ce924e56ad7.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b3f7dbc836b87d824593.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B-Instruct","Context budget (B\_total)":"1024L"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.