# Language Models Can Explain Visual Features via Steering

Source: [Language Models Can Explain Visual Features via Steering](<https://www.sota2.com/research/paper/69c36bfa0636e5bf4afc6406>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-27T05:00:04.194Z

Authors: Javier Ferrando, Enrique Lopez-Cuena, Pablo Agustin Martin-Torres, Daniel Hinjos, Anna Arias-Duart, Dario Garcia-Gasulla

Published: 2026-03-23T21:37:00.000Z

## Abstract

Sparse Autoencoders uncover thousands of features in vision models, yet explaining these features without requiring human intervention remains an open challenge. While previous work has proposed generating correlation-based explanations based on top activating input examples, we present a fundamentally different alternative based on causal interventions. We leverage the structure of Vision-Language Models and steer individual SAE features in the vision encoder after providing an empty image. Then, we prompt the language model to explain what it \`\`sees'', effectively eliciting the visual concept represented by each feature. Results show that Steering offers an scalable alternative that complements traditional approaches based on input examples, serving as a new axis for automated interpretability in vision models. Moreover, the quality of explanations improves consistently with the scale of the language model, highlighting our method as a promising direction for future research. Finally, we propose Steering-informed Top-k, a hybrid approach that combines the strengths of causal interventions and input-based approaches to achieve state-of-the-art explanation quality without additional computational cost.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [SAE Feature Explanation Evaluation on Vision Encoder SAE Features Synthetic Image Based](<https://www.sota2.com/research/sota/sae-feature-explanation-evaluation-on-vision-encoder-sae-features-synthetic-image-based.md>): SAE Feature Explanation Evaluation; Vision Encoder SAE Features Synthetic Image Based; IoU (Masks): 22.8.

- [Explanation Evaluation on Gemma Vision Encoder SAE 3 (test)](<https://www.sota2.com/research/sota/explanation-evaluation-on-gemma-vision-encoder-sae-3-test.md>): Explanation Evaluation; Gemma vision encoder later layer SAE 3 (test); IoU (Masks): 20.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.22593>)

[PDF](<https://arxiv.org/pdf/2603.22593.pdf>)