# FaithfulSAE: Towards Capturing Faithful Features with Sparse Autoencoders without External Dataset Dependencies

Source: [FaithfulSAE: Towards Capturing Faithful Features with Sparse Autoencoders without External Dataset Dependencies](<https://www.sota2.com/research/paper/68fa798d1e209131cea7e2ba>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:00:46.441Z

Authors: Seonglae Cho, Harryn Oh, Donghyun Lee, Luis Eduardo Rodrigues Vieira, Andrew Bermingham, Ziad El Sayed

Published: 2025-06-21T10:18:25.000Z

## Abstract

Sparse Autoencoders (SAEs) have emerged as a promising solution for decomposing large language model representations into interpretable features. However, Paulo and Belrose (2025) have highlighted instability across different initialization seeds, and Heap et al. (2025) have pointed out that SAEs may not capture model-internal features. These problems likely stem from training SAEs on external datasets - either collected from the Web or generated by another model - which may contain out-of-distribution (OOD) data beyond the model's generalisation capabilities. This can result in hallucinated SAE features, which we term "Fake Features", that misrepresent the model's internal activations. To address these issues, we propose FaithfulSAE, a method that trains SAEs on the model's own synthetic dataset. Using FaithfulSAEs, we demonstrate that training SAEs on less-OOD instruction datasets results in SAEs being more stable across seeds. Notably, FaithfulSAEs outperform SAEs trained on web-based datasets in the SAE probing task and exhibit a lower Fake Feature Ratio in 5 out of 7 models. Overall, our approach eliminates the dependency on external datasets, advancing interpretability by better capturing model-internal features while highlighting the often neglected importance of SAE training datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Causal Faithfulness Evaluation on FineWeb Temporal](<https://www.sota2.com/research/sota/causal-faithfulness-evaluation-on-fineweb-temporal.md>): Causal Faithfulness Evaluation; FineWeb Temporal; nAOPC: 0.982.

- [Causal Faithfulness Evaluation on HaluEval Adversarial](<https://www.sota2.com/research/sota/causal-faithfulness-evaluation-on-halueval-adversarial.md>): Causal Faithfulness Evaluation; HaluEval Adversarial; nAOPC: 100.

- [Causal Faithfulness Evaluation on Edgar Domain](<https://www.sota2.com/research/sota/causal-faithfulness-evaluation-on-edgar-domain.md>): Causal Faithfulness Evaluation; Edgar Domain; nAOPC: 0.976.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2506.17673>)

[PDF](<https://arxiv.org/pdf/2506.17673.pdf>)