Machine Translation on WMT En-De 2019 (test)
93SacreBLEUMicrosoft-WMT19-sent-doc
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Microsoft-WMT19-sent-docResource usage=Unconstrained2019.07 | 93 | 0.311 | |
| Microsoft-WMT19-doc-levelResource usage=Unconstrained2019.07 | 92.6 | 0.296 | |
| Facebook-FAIRResource usage=Constrained2019.07 | 90.3 | 0.347 | |
| HUMANResource usage=Unconstrained2019.07 | 90.3 | 0.24 | |
| NEUResource usage=Constrained2019.07 | 89.6 | 0.208 | |
| UCAMResource usage=Constrained2019.07 | 88.7 | 0.213 | |
| MSRA-MADLResource usage=Unconstrained2019.07 | 87.6 | 0.214 | |
| MLLP-UPVResource usage=Constrained2019.07 | 87.5 | 0.189 | |
| eTranslationResource usage=Constrained2019.07 | 87.5 | 0.13 | |
| TartuNLP-cResource usage=Constrained2019.07 | 87.4 | -0.132 | |
| JHUResource usage=Constrained2019.07 | 87.3 | 0.081 | |
| dfki-nmtResource usage=Constrained2019.07 | 86.8 | 0.119 | |
| Microsoft-WMT19-sent-levelResource usage=Unconstrained2019.07 | 86.6 | 0.094 | |
| Helsinki-NLPResource usage=Constrained2019.07 | 84.4 | 0.077 | |
| online-BResource usage=Unconstrained2019.07 | 84.2 | 0.094 | |
| online-YResource usage=Unconstrained2019.07 | 84.2 | 0.038 | |
| PROMT-NMTResource usage=Constrained2019.07 | 84.1 | 0.001 | |
| lmu-ctx-tf-singleResource usage=Constrained2019.07 | 83.7 | 0.01 | |
| online-AResource usage=Unconstrained2019.07 | 82.8 | -0.072 | |
| online-GResource usage=Unconstrained2019.07 | 82.7 | -0.119 | |
| UdS-DFKIResource usage=Constrained2019.07 | 80.3 | -0.129 | |
| online-XResource usage=Unconstrained2019.07 | 76.3 | -0.4 | |
| en-de-taskResource usage=Constrained2019.07 | 43.3 | -1.769 | |
| HS-NASSearch Strategy=GPT-4, HAT, 1, 15, Latency (ms)=71.9, GFLOPS=2.99, Model Size (M)=49.6, Search Hours=2.03, GFLOPs constraint=3.02023.10 | 43.1 | — | |
| HATLatency (ms)=85.5, GFLOPS=2.99, Model Size (M)=49.6, Search Hours=2.35, GFLOPs constraint=3.02023.10 | 42.9 | — | |
| UNCSAMPBackbone=TRANSFORMER-BIG, System=This Work2021.06 | 42.5 | — | |
| SRCLMBackbone=TRANSFORMER-BIG, System=This Work2021.06 | 41.7 | — | |
| RANDSAMPBackbone=TRANSFORMER-BIG, System=This Work2021.06 | 41.6 | — | |
| MATNd/d/dh=2/512/12288, #Param (M)=314.1, rho=0.22020.06 | 40.4 | — | |
| MATNd/d/dh=2/512/12288, #Param (M)=314.1, rho=0.32020.06 | 40.3 | — | |
| Wu et al. (2019b)Data=+RANDSAMP2021.06 | 39.8 | — | |
| BITEXTBackbone=TRANSFORMER-BIG, System=This Work2021.06 | 39.6 | — | |
| MATNd/d/dh=2/512/12288, #Param (M)=314.1, rho=0.12020.06 | 39.5 | — | |
| TransformerNd/d/dh=1/1024/4096, #Param (M)=325.7, rho=02020.06 | 39.3 | — | |
| MATNd/d/dh=2/512/12288, #Param (M)=314.1, rho=02020.06 | 38.5 | — | |
| TransformerNd/d/dh=1/512/12288, #Param (M)=288.9, rho=02020.06 | 37.4 | — | |
| Wu et al. (2019b)Data=BITEXT2021.06 | 37.3 | — |