# Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift

Source: [Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift](<https://www.sota2.com/research/paper/6a0fe320f984739ed6739e05>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-23T05:00:04.712Z

Authors: Sungjun Lim, Heedong Kim, Andrew Lee, Kyungwoo Song

Published: 2026-05-21T00:46:01.000Z

## Abstract

Mechanistic interpretability aims to explain a model's behavior by identifying causally responsible internal structures. Dictionary-based explainers such as sparse autoencoders and transcoders are a primary tool, but their faithfulness under out-of-distribution (OOD) shift has received little systematic attention. We show that distribution shift rotates the subspace that the model actively uses, misaligning the explainer's dictionary trained on in-distribution (ID) activations. We formalize this misalignment as the faithfulness gap, a geometric distance between the ID dictionary and the OOD-active subspace, and show that it controls OOD faithfulness degradation. To reduce this gap, we propose the Geometry-Adaptive Explainer (GAE), which realigns the explainer's dictionary with the OOD-active subspace while preserving the original feature structure. This requires only unlabeled OOD activations and no gradient updates. We prove that GAE improves over the unadapted ID explainer, with excess loss bounded quadratically by the second-moment shift. Empirically, GAE even matches or surpasses all training-based baselines in causal faithfulness across multiple models and OOD settings.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Causal Faithfulness Evaluation on FineWeb Temporal](<https://www.sota2.com/research/sota/causal-faithfulness-evaluation-on-fineweb-temporal.md>): Causal Faithfulness Evaluation; FineWeb Temporal; nAOPC: 0.985.

- [Causal Faithfulness Evaluation on Edgar Domain](<https://www.sota2.com/research/sota/causal-faithfulness-evaluation-on-edgar-domain.md>): Causal Faithfulness Evaluation; Edgar Domain; nAOPC: 0.988.

- [Causal Faithfulness Evaluation on HaluEval Adversarial](<https://www.sota2.com/research/sota/causal-faithfulness-evaluation-on-halueval-adversarial.md>): Causal Faithfulness Evaluation; HaluEval Adversarial; nAOPC: 100.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.21849>)

[PDF](<https://arxiv.org/pdf/2605.21849.pdf>)