# Long-context modeling on RULER Source: [Long-context modeling on RULER]() Collection index: [llms.txt]() Data updated: 2026-04-23T09:25:23.904Z Evaluation of long-context modeling performance on the RULER dataset, reporting results across multiple specific metrics covering various context lengths (1K, 2K, 4K, 8K) and sub-tasks (MK, MQ, CWE, Hotpot). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-context modeling | | Dataset (including any stated split) | RULER | | Metric | NIAH Score (MK, 1K) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 8 of 8 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [NIAH Score (MK, 1K)](): Higher is better; unit: PERCENT. - [NIAH Score (MK, 2K)](): Higher is better; unit: PERCENT. - [NIAH Score (MK, 4K)](): Higher is better; unit: PERCENT. - [NIAH Score (MQ, 1K)](): Higher is better; unit: PERCENT. - [NIAH Score (MQ, 2K)](): Higher is better; unit: PERCENT. - [NIAH Score (MQ, 4K)](): Higher is better; unit: PERCENT. - [RULER Score (CWE, 1K)](): Higher is better; unit: PERCENT. - [RULER Score (CWE, 2K)](): Higher is better; unit: PERCENT. - [RULER Score (CWE, 4K)](): Higher is better; unit: PERCENT. - [RULER Score (Hotpot, 2K)](): Higher is better; unit: PERCENT. - [RULER Score (Hotpot, 4K)](): Higher is better; unit: PERCENT. - [RULER Score (Hotpot, 8K)](): Higher is better; unit: PERCENT. - [RULER Score (Overall)](): Higher is better; unit: PERCENT. - [Average Score (All)](): Higher is better; unit: PERCENT. ## Recorded results | Method | NIAH Score (MK, 1K) | Result date | Report paper | | --- | --- | --- | --- | | Transformer | 90.2 | Not recorded | 68fa7c101e209131cea8c392 | | NHA | 81.4 | Not recorded | 68fa7c101e209131cea8c392 | | GDN-H | 59.6 | Not recorded | 68fa7c101e209131cea8c392 | | GLA-H | 46 | Not recorded | 68fa7c101e209131cea8c392 | | GSA-H | 20 | Not recorded | 68fa7c101e209131cea8c392 | | Mamba2-H | 13.6 | Not recorded | 68fa7c101e209131cea8c392 | ### Transformer NIAH Score (MK, 1K): 90.2. Result ID: 69e0898a47f34ce924c94d4b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training context length":"2K"}. This record does not assert independent reproduction. ### NHA NIAH Score (MK, 1K): 81.4. Result ID: 69e0898a47f34ce924c94d50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training context length":"2K"}. This record does not assert independent reproduction. ### GDN-H NIAH Score (MK, 1K): 59.6. Result ID: 69e0898a47f34ce924c94d4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training context length":"2K"}. This record does not assert independent reproduction. ### GLA-H NIAH Score (MK, 1K): 46. Result ID: 69e0898a47f34ce924c94d4d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training context length":"2K"}. This record does not assert independent reproduction. ### GSA-H NIAH Score (MK, 1K): 20. Result ID: 69e0898a47f34ce924c94d4e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training context length":"2K"}. This record does not assert independent reproduction. ### Mamba2-H NIAH Score (MK, 1K): 13.6. Result ID: 69e0898a47f34ce924c94d4c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training context length":"2K"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.