# Formality Style Transfer on GYAFC Family & Relationships 1.0 (test)

Source: [Formality Style Transfer on GYAFC Family & Relationships 1.0 (test)](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-1-0-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:36:49.561Z

Evaluation of models performing Formality Style Transfer on the test split of the GYAFC Family & Relationships 1.0 dataset, utilizing BLEURT, BLEU, ACC, and HM metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Formality Style Transfer |
| Dataset (including any stated split) | GYAFC Family & Relationships 1.0 (test) |
| Metric | BLEU |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [BLEU](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-1-0-test.md?metric=BLEU>): Higher is better; unit: SCALAR.

- [BLEURT](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-1-0-test.md?metric=BLEURT>): Higher is better; unit: SCALAR.

- [Acc](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-1-0-test.md?metric=Acc>): Higher is better; unit: PERCENT.

- [HM](<https://www.sota2.com/research/sota/formality-style-transfer-on-gyafc-family-and-relationships-1-0-test.md?metric=HM>): Higher is better; unit: SCALAR.

## Recorded results

| Method | BLEU | Result date | Report paper |
| --- | --- | --- | --- |
| Chawla's | 0.799 | Not recorded | 68fa6a571e209131cea27925 |
| BART large + SC & BLEU | 0.793 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | 0.777 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | 0.77 | Not recorded | 68fa6a571e209131cea27925 |
| GPT-CAT | 0.769 | Not recorded | 68fa6a571e209131cea27925 |
| BART large + SC & BLEU | 0.611 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | 0.572 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | 0.571 | Not recorded | 68fa6a571e209131cea27925 |
| BART + SC & BLEU | 0.571 | Not recorded | 68fa6a571e209131cea27925 |
| Bi-directional FT | 0.568 | Not recorded | 68fa6a571e209131cea27925 |
| GPT-2 + SC & BLEU | 0.528 | Not recorded | 68fa6a571e209131cea27925 |
| GPT-2 + SC & BLEU | 0.527 | Not recorded | 68fa6a571e209131cea27925 |
| NMT-Combined | 0.477 | Not recorded | 68fa6a571e209131cea27925 |
| OpenNMT + SC & BLEU | 0.403 | Not recorded | 68fa6a571e209131cea27925 |
| CPLS | 0.379 | Not recorded | 68fa6a571e209131cea27925 |

### Chawla's

BLEU: 0.799. Result ID: 698e2d0affc3901de09b68d6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68851e209131cea1caa6.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### BART large + SC & BLEU

BLEU: 0.793. Result ID: 698e2d0affc3901de09b68d9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Domain Context":"Combined (F&R and E&M)","Model Architecture":"BART large"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEU: 0.777. Result ID: 698e2d0affc3901de09b68d8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Domain Context":"Combined (F&R and E&M)","Model Architecture":"BART"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEU: 0.77. Result ID: 698e2d0affc3901de09b68d7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Domain Context":"Single domain (F&R)","Model Architecture":"BART"}. This record does not assert independent reproduction.

### GPT-CAT

BLEU: 0.769. Result ID: 698e2d0affc3901de09b68d5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

Recorded evaluation configuration: {"Training Domain Context":"Combined (E&M and F&R)","Model Architecture":"GPT-2"}. This record does not assert independent reproduction.

### BART large + SC & BLEU

BLEU: 0.611. Result ID: 698e2d0affc3901de09b68db.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"100%","Training Domain Context":"Combined Domains","Model Architecture":"BART large"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEU: 0.572. Result ID: 698e2d0affc3901de09b68d3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"10%","Model Architecture":"BART"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEU: 0.571. Result ID: 698e2d0affc3901de09b68d4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"100%","Model Architecture":"BART"}. This record does not assert independent reproduction.

### BART + SC & BLEU

BLEU: 0.571. Result ID: 698e2d0affc3901de09b68dd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"10% Parallel Training Data","Model Architecture":"BART"}. This record does not assert independent reproduction.

### Bi-directional FT

BLEU: 0.568. Result ID: 698e2d0affc3901de09b68da.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64811e209131cea04f0b.md>)

Recorded evaluation configuration: {"Training Domain Context":"Combined Domains"}. This record does not assert independent reproduction.

### GPT-2 + SC & BLEU

BLEU: 0.528. Result ID: 698e2d0affc3901de09b68d2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"100%","Model Architecture":"GPT-2"}. This record does not assert independent reproduction.

### GPT-2 + SC & BLEU

BLEU: 0.527. Result ID: 698e2d0affc3901de09b68d1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Implementation](<https://github.com/laihuiyuan/Pre-trained-formality-transfer>)

Recorded evaluation configuration: {"Training Data Volume":"10%","Model Architecture":"GPT-2"}. This record does not assert independent reproduction.

### NMT-Combined

BLEU: 0.477. Result ID: 698e2d0affc3901de09b68d0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64491e209131cea03b73.md>)

Recorded evaluation configuration: {"Model Architecture":"Transformer"}. This record does not assert independent reproduction.

### OpenNMT + SC & BLEU

BLEU: 0.403. Result ID: 698e2d0affc3901de09b68cf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63341e209131ce9fd60c.md>)

Recorded evaluation configuration: {"Training Data Volume":"100%","Model Architecture":"OpenNMT"}. This record does not assert independent reproduction.

### CPLS

BLEU: 0.379. Result ID: 698e2d0affc3901de09b68dc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6a571e209131cea27925.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66761e209131cea10473.md>)

Recorded evaluation configuration: {"Training Data Volume":"10% Parallel Training Data"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
