Machine Translation on WMT En-Ro 2016 (test)
35.4BLEUEIT deep
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| EIT deepType=Our System, Depth=24-6, Parameters (M)=111.092022.12 | 35.4 | 82.46 | — | — | — | — | — | |
| E-EITType=Our System, Depth=24-6, Parameters (M)=110.732022.12 | 35.35 | 82.55 | — | — | — | — | — | |
| EIT baseType=Our System, Depth=6-6, Parameters (M)=53.982022.12 | 35.1 | 82.18 | — | — | — | — | — | |
| E-EIT baseType=Our System, Depth=6-6, Parameters (M)=53.922022.12 | 35.01 | 82.05 | — | — | — | — | — | |
| Transformer deepType=Our System, Depth=24-6, Parameters (M)=110.642022.12 | 35 | 82.11 | — | — | — | — | — | |
| EIT bigType=Our System, Depth=6-6, Parameters (M)=196.42022.12 | 34.91 | 82.15 | — | — | — | — | — | |
| UMSTType=Localness, Depth=6-6, Parameters (M)=602022.12 | 34.81 | — | — | — | — | — | — | |
| DelightType=Other Baselines, Parameters (M)=532022.12 | 34.7 | — | — | — | — | — | — | |
| TIE-KDBase architecture=Transformer-base, Learning type=Knowledge Distillation2023.05 | 34.7 | 55.76 | — | — | — | — | — | |
| Transformer-bigRole=Teacher2023.05 | 34.7 | 57.04 | — | — | — | — | — | |
| E-EIT bigType=Our System, Depth=6-6, Parameters (M)=195.972022.12 | 34.67 | 81.8 | — | — | — | — | — | |
| CollaborationType=Multi-Head, Depth=6-6, Parameters (M)=542022.12 | 34.64 | — | — | — | — | — | — | |
| DMANType=Localness2022.12 | 34.49 | — | — | — | — | — | — | |
| Transformer bigType=Our System, Depth=6-6, Parameters (M)=195.882022.12 | 34.44 | 81.63 | — | — | — | — | — | |
| FISHformerType=Multi-Head, Depth=6-6, Parameters (M)=492022.12 | 34.42 | — | — | — | — | — | — | |
| MoAType=Multi-Head, Depth=6-6, Parameters (M)=562022.12 | 34.39 | — | — | — | — | — | — | |
| ATKnowledge Distillation=w/o2024.05 | 34.39 | — | 58.48 | 78.9 | 69.89 | 86.18 | 1 | |
| Talking-HeadType=Multi-Head, Depth=6-6, Parameters (M)=542022.12 | 34.35 | — | — | — | — | — | — | |
| Transformer in Liu et al. (2020)Type=Other Baselines2022.12 | 34.3 | — | — | — | — | — | — | |
| RefinerType=Multi-Head, Depth=6-6, Parameters (M)=542022.12 | 34.25 | — | — | — | — | — | — | |
| Transformer baseType=Our System, Depth=6-6, Parameters (M)=53.92022.12 | 34.23 | 81.39 | — | — | — | — | — | |
| Transformer in Kasai et al. (2020)Type=Other Baselines2022.12 | 34.16 | — | — | — | — | — | — | |
| ATKnowledge Distillation=w/2024.05 | 33.92 | — | 58.45 | 78.49 | 69.26 | 86.22 | 1 | |
| Word-KDBase architecture=Transformer-base, Learning type=Knowledge Distillation2023.05 | 33.77 | 53.15 | — | — | — | — | — | |
| Seer ForcingBase architecture=Transformer-base, Learning type=Knowledge Distillation2023.05 | 33.77 | 51.41 | — | — | — | — | — | |
| Selective-KDBase architecture=Transformer-base, Learning type=Knowledge Distillation2023.05 | 33.74 | 53.05 | — | — | — | — | — | |
| Seq-KDBase architecture=Transformer-base, Learning type=Knowledge Distillation2023.05 | 33.69 | 50.63 | — | — | — | — | — | |
| Annealing KDBase architecture=Transformer-base, Learning type=Knowledge Distillation2023.05 | 33.67 | 52.22 | — | — | — | — | — | |
| Transformer-baseRole=Student (Baseline)2023.05 | 33.59 | 50.96 | — | — | — | — | — | |
| CTCKnowledge Distillation=w/2024.05 | 33.28 | — | 58.28 | 75.54 | 65.71 | 82.94 | 14.5 | |
| DATKnowledge Distillation=w/2024.05 | 33.25 | — | 57.89 | 76.59 | 67.01 | 84.27 | 13.7 | |
| DATKnowledge Distillation=w/o2024.05 | 33.18 | — | 57.35 | 76.14 | 66.72 | 83.72 | 13.8 | |
| MgMOKnowledge Distillation=w/2024.05 | 32.86 | — | 57.4 | 75.52 | 65.36 | 82.73 | 14.9 | |
| CTCKnowledge Distillation=w/o2024.05 | 32.73 | — | 57.77 | 74.26 | 63.99 | 81.16 | 14.5 | |
| CMLMKnowledge Distillation=w/2024.05 | 32.71 | — | 56.76 | 72.36 | 63.42 | 76.67 | 2.7 | |
| CMLMKnowledge Distillation=w/o2024.05 | 31.97 | — | 56.78 | 74.11 | 63.34 | 78.72 | 2.7 | |
| MgMOKnowledge Distillation=w/o2024.05 | 30.97 | — | 56.65 | 73.54 | 63.19 | 80.04 | 14.9 | |
| NATKnowledge Distillation=w/2024.05 | 30.97 | — | 56.52 | 72.6 | 62.12 | 77.47 | 15.8 | |
| NATKnowledge Distillation=w/o2024.05 | 23.75 | — | 50.72 | 65.78 | 53.91 | 67.29 | 15.9 |