# Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency

Source: [Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency](<https://www.sota2.com/research/paper/69bb82ed0636e5bf4afc18be>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-20T05:00:04.947Z

Authors: Lucas Bandarkar, Alan Ansell, Trevor Cohn

Published: 2026-03-17T19:48:44.000Z

## Abstract

Modern LLMs continue to exhibit significant variance in behavior across languages, such as being able to recall factual information in some languages but not others. While typically studied as a problem to be mitigated, in this work, we propose leveraging this cross-lingual inconsistency as a tool for interpretability in mixture-of-experts (MoE) LLMs. Our knowledge localization framework contrasts routing for sets of languages where the model correctly recalls information from languages where it fails. This allows us to isolate model components that play a functional role in answering about a piece of knowledge. Our method proceeds in two stages: (1) querying the model with difficult factual questions across a diverse set of languages to generate "success" and "failure" activation buckets and then (2) applying a statistical contrastive analysis to the MoE router logits to identify experts important for knowledge. To validate the necessity of this small number of experts for answering a knowledge question, we deactivate them and re-ask the question. We find that despite only deactivating about 20 out of 6000 experts, the model no longer answers correctly in over 40% of cases. Generally, this method provides a realistic and scalable knowledge localization approach to address increasingly complex LLMs.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Knowledge Attribution Causal Ablation on ECLeKTic (test)](<https://www.sota2.com/research/sota/knowledge-attribution-causal-ablation-on-eclektic-test.md>): Knowledge Attribution Causal Ablation; ECLeKTic (test); Ablation Success Rate: 43.7.

- [Knowledge Attribution Causal Ablation on MultiLoKo (test)](<https://www.sota2.com/research/sota/knowledge-attribution-causal-ablation-on-multiloko-test.md>): Knowledge Attribution Causal Ablation; MultiLoKo (test); Ablation Success Rate: 50.1.

- [Knowledge Attribution Causal Ablation on G-MMLU (test)](<https://www.sota2.com/research/sota/knowledge-attribution-causal-ablation-on-g-mmlu-test.md>): Knowledge Attribution Causal Ablation; G-MMLU (test); Ablation Success Rate: 47.5.

- [Knowledge Attribution Causal Ablation on Total (Combined ECLeKTic, MultiLoKo, G-MMLU) (test)](<https://www.sota2.com/research/sota/knowledge-attribution-causal-ablation-on-total-combined-eclektic-multiloko-g-mmlu-test.md>): Knowledge Attribution Causal Ablation; Total (Combined ECLeKTic, MultiLoKo, G-MMLU) (test); Ablation Success Rate: 48.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.17102>)

[PDF](<https://arxiv.org/pdf/2603.17102.pdf>)
