ResearchBenchmarksLong-context language modeling on RULER (32K Context)Long-context language modeling on RULER (32K Context)Follow100S-NIAH Component 1 ScoreRDKV99.79299.84699.999.954May 8, 2026Evaluation resultsMethodMethodLinksS-NIAH Component 1 ScoreS-NIAH Component 2 ScoreS-NIAH Component 3 ScoreMK-NIAH Component 1 ScoreMK-NIAH Component 2 ScoreMK-NIAH Component 3 ScoreMQ-NIAH ScoreMV-NIAH ScoreVT ScoreCWE ScoreFWE ScoreAverage ScoreRDKVBackbone=LLaMA-3.1-8B-...Backbone=LLaMA-3.1-8B-Instruct, KV cache budget (Btotal)=1024L2026.0510098.499.698.498.871.299.298.398.737.371.288.3HqeKVBackbone=LLaMA-3.1-8B-...Backbone=LLaMA-3.1-8B-Instruct, KV cache budget (Btotal)=1024L2026.0599.899.288.896.698.685.293.89195.315.387.286.4