# Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization

Source: [Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization](<https://www.sota2.com/research/paper/68fa68851e209131cea1caa6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:56:44.875Z

Authors: Kunal Chawla, Diyi Yang

Published: 2020-10-10T21:05:56.000Z

## Abstract

Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences, which can be used to improve performance of many downstream NLP tasks. In this work, we propose a semi-supervised formality style transfer model that utilizes a language model-based discriminator to maximize the likelihood of the output sentence being formal, which allows us to use maximization of token-level conditional probabilities for training. We further propose to maximize mutual information between source and target styles as our training objective instead of maximizing the regular likelihood that often leads to repetitive and trivial generated responses. Experiments showed that our model outperformed previous state-of-the-art baselines significantly in terms of both automated metrics and human judgement. We further generalized our model to unsupervised text style transfer task, and achieved significant improvements on two benchmark sentiment style transfer datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Formality Style Transfer on GYAFC Family & Relationships 1.0 (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-1-0-test.md>): Formality Style Transfer; GYAFC Family & Relationships 1.0 (test); BLEU: 0.799.

- [Formality Style Transfer on GYAFC Entertainment & Music 1.0 (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-1-0-test.md>): Formality Style Transfer; GYAFC Entertainment & Music 1.0 (test); BLEURT: 0.26.

- [Formality Style Transfer on GYAFC (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-test.md>): Formality Style Transfer; GYAFC (test); Formality (E&M): 1.46.

- [Formality Style Transfer on GYAFC Family & Relationships (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-test.md>): Formality Style Transfer; GYAFC Family & Relationships (test); BLEU: 79.92.

- [Formality Style Transfer on GYAFC Entertainment & Music (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-test.md>): Formality Style Transfer; GYAFC Entertainment & Music (test); BLEU: 76.17.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2010.05090>)

[PDF](<https://arxiv.org/pdf/2010.05090.pdf>)
