# Why Does New Knowledge Create Messy Ripple Effects in LLMs? Source: [Why Does New Knowledge Create Messy Ripple Effects in LLMs?]() Collection index: [llms.txt]() Data updated: 2025-10-26T05:12:23.158Z Authors: Jiaxin Qin, Zixuan Zhang, Manling Li, Pengfei Yu, Heng Ji Published: 2024-07-02T14:33:44.000Z ## Abstract Extensive previous research has focused on post-training knowledge editing (KE) for language models (LMs) to ensure that knowledge remains accurate and up-to-date. One desired property and open question in KE is to let edited LMs correctly handle ripple effects, where LM is expected to answer its logically related knowledge accurately. In this paper, we answer the question of why most KE methods still create messy ripple effects. We conduct extensive analysis and identify a salient indicator, GradSim, that effectively reveals when and why updated knowledge ripples in LMs. GradSim is computed by the cosine similarity between gradients of the original fact and its related knowledge. We observe a strong positive correlation between ripple effect performance and GradSim across different LMs, KE methods, and evaluation metrics. Further investigations into three counter-intuitive failure cases (Negation, Over-Ripple, Multi-Lingual) of ripple effects demonstrate that these failures are often associated with very low GradSim. This finding validates that GradSim is an effective indicator of when knowledge ripples in LMs. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Efficiency Evaluation of Factual Representation Extraction on Corpus of 11,427 facts](): Efficiency evaluation of factual representation extraction; Corpus of 11,427 facts; Total Processing Time (s): 0.763. - [Ripple effect magnitude prediction (\|Δ log P(y)\|) on Curated ripple-effect corpus 1.0 (test)](): Ripple effect magnitude prediction (\|Δ log P(y)\|); Curated ripple-effect corpus 1.0 (test); Spearman Correlation (ρs): 0.597. - [Ripple effect magnitude prediction (ℓ2 logit shift) on Curated ripple-effect corpus 1.0 (test)](): Ripple effect magnitude prediction (ℓ2 logit shift); Curated ripple-effect corpus 1.0 (test); Spearman Correlation (ρs): 0.691. ## Original sources and implementation [Original paper]() [PDF]()