ResearchBenchmarksLong-context language modeling on RULER Sequence length = 16kLong-context language modeling on RULER Sequence length = 16kFollow100S-NIAH Component 1HqeKV9597.5100102.5May 8, 2026Evaluation resultsMethodMethodLinksS-NIAH Component 1S-NIAH Component 2S-NIAH Component 3MK-NIAH Component 1MK-NIAH Component 2MK-NIAH Component 3MQ-NIAH ScoreMV-NIAH ScoreVT ScoreCWE ScoreFWE ScoreAverage ScoreHqeKVBackbone=LLaMA-3.1-8B-...Backbone=LLaMA-3.1-8B-Instruct, KV cache budget (Btotal)=1024L2026.0510099.49999.699.886.496.795.196.226.89590.4RDKVBackbone=LLaMA-3.1-8B-...Backbone=LLaMA-3.1-8B-Instruct, KV cache budget (Btotal)=1024L2026.0510099.810099.699.686.299.498.599.276.793.295.7