# Who's asking? User personas and the mechanics of latent misalignment

Source: [Who's asking? User personas and the mechanics of latent misalignment](<https://www.sota2.com/research/paper/68fa746c1e209131cea619fe>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:38:02.139Z

Authors: Asma Ghandeharioun, Ann Yuan, Marius Guerard, Emily Reif, Michael A. Lepori, Lucas Dixon

Published: 2024-06-17T21:15:12.000Z

## Abstract

Despite investments in improving model safety, studies show that misaligned capabilities remain latent in safety-tuned models. In this work, we shed light on the mechanics of this phenomenon. First, we show that even when model generations are safe, harmful content can persist in hidden representations and can be extracted by decoding from earlier layers. Then, we show that whether the model divulges such content depends significantly on its perception of who it is talking to, which we refer to as user persona. In fact, we find manipulating user persona to be even more effective for eliciting harmful content than direct attempts to control model refusal. We study both natural language prompting and activation steering as control methods and show that activation steering is significantly more effective at bypassing safety filters. We investigate why certain personas break model safeguards and find that they enable the model to form more charitable interpretations of otherwise dangerous queries. Finally, we show we can predict a persona's effect on refusal given only the geometry of its steering vector.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Cross-lingual Cultural Consistency on BLEnD All 8 Languages](<https://www.sota2.com/research/sota/cross-lingual-cultural-consistency-on-blend-all-8-languages.md>): Cross-lingual Cultural Consistency; BLEnD All 8 Languages; Max Sigma: 0.017.

- [Cross-lingual Cultural Consistency on BLEnD Higher-Resource](<https://www.sota2.com/research/sota/cross-lingual-cultural-consistency-on-blend-higher-resource.md>): Cross-lingual Cultural Consistency; BLEnD Higher-Resource; Max Sigma: 0.025.

- [Cross-lingual Cultural Consistency on BLEnD Lower-Resource](<https://www.sota2.com/research/sota/cross-lingual-cultural-consistency-on-blend-lower-resource.md>): Cross-lingual Cultural Consistency; BLEnD Lower-Resource; Max Sigma: 0.02.

- [Cross-lingual Cultural Consistency on BLEnD Indo-European](<https://www.sota2.com/research/sota/cross-lingual-cultural-consistency-on-blend-indo-european.md>): Cross-lingual Cultural Consistency; BLEnD Indo-European; Max Sigma: 0.021.

- [Cross-lingual Cultural Consistency on BLEnD Non-Indo-European](<https://www.sota2.com/research/sota/cross-lingual-cultural-consistency-on-blend-non-indo-european.md>): Cross-lingual Cultural Consistency; BLEnD Non-Indo-European; Max Sigma: 0.022.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2406.12094>)

[PDF](<https://arxiv.org/pdf/2406.12094.pdf>)
