# Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer

Source: [Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:05:13.543Z

Authors: Huiyuan Lai, Antonio Toral, Malvina Nissim

Published: 2021-05-14T16:39:22.000Z

## Abstract

Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content. We show that fine-tuning pre-trained language (GPT-2) and sequence-to-sequence (BART) models boosts content preservation, and that this is possible even with limited amounts of parallel data. Augmenting these models with rewards that target style and content -- the two core aspects of the task -- we achieve a new state-of-the-art.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Formality Style Transfer on GYAFC Entertainment & Music 1.0 (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-1-0-test.md>): Formality Style Transfer; GYAFC Entertainment & Music 1.0 (test); BLEURT: 0.274.

- [Formality Style Transfer on GYAFC Family & Relationships 1.0 (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-1-0-test.md>): Formality Style Transfer; GYAFC Family & Relationships 1.0 (test); BLEU: 0.793.

- [Formality Style Transfer on GYAFC Entertainment & Music (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-test.md>): Formality Style Transfer; GYAFC Entertainment & Music (test); BLEU: 76.5.

- [Formality Style Transfer on GYAFC Family & Relationships (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-test.md>): Formality Style Transfer; GYAFC Family & Relationships (test); BLEU: 79.25.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2105.06947>)

[PDF](<https://arxiv.org/pdf/2105.06947.pdf>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)
