Machine Translation on WMT English-German (EN-DE) 2014 (newstest2014)
30.77BLEURK2-block (learnable gamma_i)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RK2-block (learnable gamma_i)Layers=12-6, #Param=286M, Steps=100K2022.03 | 30.77 | 29.6 | |
| BERT-fused modelLayers=6-62022.03 | 30.75 | — | |
| RK2-blockLayers=12-6, #Param=286M, Steps=100K2022.03 | 30.58 | 29.4 | |
| Multiscale CollaborativeLayers=18-6, #Param=512M, Steps=300K2022.03 | 30.56 | — | |
| RK4-blockLayers=12-6, #Param=286M, Steps=100K2022.03 | 30.55 | 29.4 | |
| RK2-block (learnable gamma_i)Layers=6-6, #Param=211M, Steps=100K2022.03 | 30.53 | 29.4 | |
| RK4-blockLayers=6-6, #Param=211M, Steps=100K2022.03 | 30.39 | 29.3 | |
| RK2-block (learnable gamma_i)Layers=24-6, #Param=118M, Steps=50K2022.03 | 30.29 | 29.2 | |
| SDTLayers=48-6, #Param=192M, Steps=50K2022.03 | 30.21 | 29 | |
| MacaronNetLayers=6-62022.03 | 30.2 | — | |
| RK2-blockLayers=6-6, #Param=211M, Steps=100K2022.03 | 30.11 | 29 | |
| ADMINLayers=60-12, #Param=262M, Steps=250K2022.03 | 30.01 | 29.5 | |
| Depth growingLayers=8-8, #Param=270M, Steps=800K2022.03 | 29.92 | — | |
| Residual-block-BigLayers=12-6, #Param=286M, Steps=100K2022.03 | 29.91 | 28.9 | |
| RK2-blockLayers=24-6, #Param=118M, Steps=50K2022.03 | 29.85 | 28.7 | |
| RK4-blockLayers=24-6, #Param=118M, Steps=50K2022.03 | 29.8 | 28.8 | |
| Residual-blockLayers=24-6, #Param=118M, Steps=50K2022.03 | 29.43 | 28.3 | |
| Transformer-DLCLLayers=30-6, #Param=137M, Steps=50K2022.03 | 29.3 | 28.6 | |
| Residual-block-BigLayers=6-6, #Param=211M, Steps=100K2022.03 | 29.21 | 28.1 | |
| RK4-blockLayers=6-6, #Param=61M, Steps=50K2022.03 | 29.03 | 27.9 | |
| RK2-block (learnable gamma_i)Layers=6-6, #Param=61M, Steps=50K2022.03 | 28.89 | 27.7 | |
| RK2-blockLayers=6-6, #Param=61M, Steps=50K2022.03 | 28.67 | 27.5 | |
| TransformerLayers=6-6, #Param=213M, Steps=100K2022.03 | 28.4 | — | |
| Residual-blockLayers=6-6, #Param=61M, Steps=50K2022.03 | 27.89 | 26.8 | |
| Transformer (base)2021.05 | 27.3 | — | |
| ResMLP-12layers=12, dimension=512, hidden MLP size=2,0482021.05 | 26.8 | — | |
| ResMLP-6layers=6, dimension=512, hidden MLP size=2,0482021.05 | 26.4 | — | |
| ConvS2S2021.05 | 25.2 | — | |
| GNMT2021.05 | 24.6 | — |