# Style-Specific Neurons for Steering LLMs in Text Style Transfer Source: [Style-Specific Neurons for Steering LLMs in Text Style Transfer]() Collection index: [llms.txt]() Data updated: 2025-10-27T06:41:00.437Z Authors: Wen Lai, Viktor Hangya, Alexander Fraser Published: 2024-10-01T11:25:36.000Z ## Abstract Text style transfer (TST) aims to modify the style of a text without altering its original meaning. Large language models (LLMs) demonstrate superior performance across multiple tasks, including TST. However, in zero-shot setups, they tend to directly copy a significant portion of the input text to the output without effectively changing its style. To enhance the stylistic variety and fluency of the text, we present sNeuron-TST, a novel approach for steering LLMs using style-specific neurons in TST. Specifically, we identify neurons associated with the source and target styles and deactivate source-style-only neurons to give target-style words a higher probability, aiming to enhance the stylistic diversity of the generated text. However, we find that this deactivation negatively impacts the fluency of the generated text, which we address by proposing an improved contrastive decoding method that accounts for rapid token probability shifts across layers caused by deactivated source-style neurons. Empirical experiments demonstrate the effectiveness of the proposed method on six benchmarks, encompassing formality, toxicity, politics, politeness, authorship, and sentiment. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Gendered-term mention ratio control on InclusiveGender](): Gendered-term mention ratio control; InclusiveGender; Δ M Mention Ratio: -2.37. - [Gendered-term mention ratio control on GCGender](): Gendered-term mention ratio control; GCGender; Delta M Mention Ratio: -0.92. - [Gender Controlled Generation on InclusiveGender (Masculine)](): Gender Controlled Generation; InclusiveGender (Masculine); Avg. Gendered Terms per Response (M): 1.87. - [Gender Neuron Masking on InclusiveGender Feminine](): Gender Neuron Masking; InclusiveGender Feminine; Count of Gendered Terms per Response (M): 0.8. - [Gender Neuron Masking on GCGender Feminine](): Gender Neuron Masking; GCGender Feminine; Term Count (M): 0.71. - [Gender Neuron Masking on InclusiveGender Masculine](): Gender Neuron Masking; InclusiveGender (Masculine); Avg. Gendered Terms per Response (M): 1.56. - [Gender Controlled Generation on InclusiveGender Feminine](): Gender Controlled Generation; InclusiveGender Feminine; Gendered Terms per Response (M): 0.1. - [Gender Controlled Generation on GCGender (Masculine)](): Gender Controlled Generation; GCGender (Masculine); Avg Count (M): 0.86. - [Gender Neuron Masking on GCGender Masculine](): Gender Neuron Masking; GCGender (Masculine); Count of Gendered Terms per Response (M): 1.01. - [Gender Controlled Generation on InclusiveGender (Neutral)](): Gender Controlled Generation; InclusiveGender (Neutral); Avg Gendered Terms per Response (M): 0.21. - [Gender Controlled Generation on GCGender (Neutral)](): Gender Controlled Generation; GCGender (Neutral); Avg Num Gendered Terms per Response (M): 0.06. - [Gender Neuron Masking on InclusiveGender Neutral](): Gender Neuron Masking; InclusiveGender (Neutral); Num. Gendered Terms per Response (M): 0.28. - [Gender Neuron Masking on GCGender Neutral](): Gender Neuron Masking; GCGender (Neutral); Term Count (M) per Response: 0.03. ## Original sources and implementation [Original paper]() [PDF]()