Formality Style Transfer on GYAFC Entertainment & Music 1.0 (test)
0.274BLEURTBART large + SC & BLEU
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| BART large + SC & BLEUTraining Domain Context=Combined (E&M and F&R), Model Architecture=BART large2021.05 | 0.274 | 0.765 | 92.9 | 0.839 | |
| Chawla's2021.05 | 0.26 | 0.762 | 91 | 0.829 | |
| BART + SC & BLEUTraining Domain Context=Combined (E&M and F&R), Model Architecture=BART2021.05 | 0.236 | 0.745 | 93.7 | 0.83 | |
| BART + SC & BLEUTraining Domain Context=Single domain (E&M), Model Architecture=BART2021.05 | 0.218 | 0.73 | 88.7 | 0.801 | |
| GPT-CATTraining Domain Context=Combined (E&M and F&R), Model Architecture=GPT-22021.05 | 0.176 | 0.725 | 87.6 | 0.793 | |
| BART large + SC & BLEUTraining Data Volume=100%, Training Domain Context=Combined Domains, Model Architecture=BART large2021.05 | 0.078 | 0.596 | 90.5 | 0.719 | |
| BART + SC & BLEUTraining Data Volume=100%, Model Architecture=BART2021.05 | 0.044 | 0.577 | 85.9 | 0.69 | |
| Bi-directional FTTraining Domain Context=Combined Domains2021.05 | 0.023 | 0.554 | 81.8 | 0.661 | |
| GPT-2 + SC & BLEUTraining Data Volume=100%, Model Architecture=GPT-22021.05 | -0.007 | 0.542 | 92.3 | 0.683 | |
| BART + SC & BLEUTraining Data Volume=10%, Model Architecture=BART2021.05 | -0.03 | 0.547 | 85.5 | 0.667 | |
| GPT-2 + SC & BLEUTraining Data Volume=10%, Model Architecture=GPT-22021.05 | -0.058 | 0.495 | 79.9 | 0.611 | |
| BART + SC & BLEUTraining Data Volume=100%, Evaluation Protocol=Evaluated against first reference only, Model Architecture=BART2021.05 | -0.089 | 0.306 | 85.9 | 0.448 | |
| NMT-CombinedModel Architecture=Transformer2021.05 | -0.1 | 0.501 | 79.7 | 0.615 | |
| TS->CPEvaluation Protocol=Evaluated against first reference only2021.05 | -0.706 | 0.292 | 59.6 | 0.392 | |
| OpenNMT + SC & BLEUTraining Data Volume=10%, Model Architecture=OpenNMT2021.05 | -0.919 | 0.231 | 88.6 | 0.366 |