# Formality Style Transfer on GYAFC Entertainment & Music 1.0 (test)

Source: [Formality Style Transfer on GYAFC Entertainment & Music 1.0 (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-1-0-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:37:22.080Z

Evaluation of models performing Formality Style Transfer on the test split of the GYAFC Entertainment & Music 1.0 dataset, utilizing BLEURT, BLEU, ACC, and HM metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Formality Style Transfer |
| Dataset (including any stated split) | GYAFC Entertainment & Music 1.0 (test) |
| Metric | BLEURT |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [BLEURT](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-1-0-test.md?metric=BLEURT>): Higher is better; unit: SCALAR.

- [BLEU](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-1-0-test.md?metric=BLEU>): Higher is better; unit: SCALAR.

- [Acc](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-1-0-test.md?metric=Acc>): Higher is better; unit: PERCENT.

- [HM](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-entertainment-and-music-1-0-test.md?metric=HM>): Higher is better; unit: SCALAR.

## Recorded results

| Method | BLEURT | Result date | Report paper |
| --- | --- | --- | --- |
| BART large + SC & BLEU | 0.274 | Not recorded | 68fa6a571e209131cea27925 |
| Chawla's | 0.26 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | 0.236 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | 0.218 | Not recorded | 68fa6a571e209131cea27925 |
| GPT-CAT | 0.176 | Not recorded | 68fa6a571e209131cea27925 |
| BART large + SC & BLEU | 0.078 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | 0.044 | Not recorded | 68fa6a571e209131cea27925 |
| Bi-directional FT | 0.023 | Not recorded | 68fa6a571e209131cea27925 |
| GPT-2 + SC & BLEU | -0.007 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | -0.03 | Not recorded | 68fa6a571e209131cea27925 |
| GPT-2 + SC & BLEU | -0.058 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | -0.089 | Not recorded | 68fa6a571e209131cea27925 |
| NMT-Combined | -0.1 | Not recorded | 68fa6a571e209131cea27925 |
| TS-\>CP | -0.706 | Not recorded | 68fa6a571e209131cea27925 |
| OpenNMT + SC & BLEU | -0.919 | Not recorded | 68fa6a571e209131cea27925 |

### BART large + SC & BLEU

BLEURT: 0.274. Result ID: 698e2d0affc3901de09b68ca.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Domain Context":"Combined (E&M and F&R)","Model Architecture":"BART large"}. This record does not assert independent reproduction.

### Chawla's

BLEURT: 0.26. Result ID: 698e2d0affc3901de09b68c7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68851e209131cea1caa6.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEURT: 0.236. Result ID: 698e2d0affc3901de09b68c9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Domain Context":"Combined (E&M and F&R)","Model Architecture":"BART"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEURT: 0.218. Result ID: 698e2d0affc3901de09b68c8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Domain Context":"Single domain (E&M)","Model Architecture":"BART"}. This record does not assert independent reproduction.

### GPT-CAT

BLEURT: 0.176. Result ID: 698e2d0affc3901de09b68c6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

Recorded evaluation configuration: {"Training Domain Context":"Combined (E&M and F&R)","Model Architecture":"GPT-2"}. This record does not assert independent reproduction.

### BART large + SC & BLEU

BLEURT: 0.078. Result ID: 698e2d0affc3901de09b68cc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"100%","Training Domain Context":"Combined Domains","Model Architecture":"BART large"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEURT: 0.044. Result ID: 698e2d0affc3901de09b68c5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"100%","Model Architecture":"BART"}. This record does not assert independent reproduction.

### Bi-directional FT

BLEURT: 0.023. Result ID: 698e2d0affc3901de09b68cb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64811e209131cea04f0b.md>)

Recorded evaluation configuration: {"Training Domain Context":"Combined Domains"}. This record does not assert independent reproduction.

### GPT-2 + SC & BLEU

BLEURT: -0.007. Result ID: 698e2d0affc3901de09b68c3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"100%","Model Architecture":"GPT-2"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEURT: -0.03. Result ID: 698e2d0affc3901de09b68c4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"10%","Model Architecture":"BART"}. This record does not assert independent reproduction.

### GPT-2 + SC & BLEU

BLEURT: -0.058. Result ID: 698e2d0affc3901de09b68c2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"10%","Model Architecture":"GPT-2"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEURT: -0.089. Result ID: 698e2d0affc3901de09b68ce.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"100%","Evaluation Protocol":"Evaluated against first reference only","Model Architecture":"BART"}. This record does not assert independent reproduction.

### NMT-Combined

BLEURT: -0.1. Result ID: 698e2d0affc3901de09b68c1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64491e209131cea03b73.md>)

Recorded evaluation configuration: {"Model Architecture":"Transformer"}. This record does not assert independent reproduction.

### TS-\>CP

BLEURT: -0.706. Result ID: 698e2d0affc3901de09b68cd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa678e1e209131cea16f61.md>)

Recorded evaluation configuration: {"Evaluation Protocol":"Evaluated against first reference only"}. This record does not assert independent reproduction.

### OpenNMT + SC & BLEU

BLEURT: -0.919. Result ID: 698e2d0affc3901de09b68c0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63341e209131ce9fd60c.md>)

Recorded evaluation configuration: {"Training Data Volume":"10%","Model Architecture":"OpenNMT"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
