Machine Translation on WMT14 DE-EN (test)
32.7BLEUTransformer
Evaluation results
| Method | Links | |
|---|---|---|
| TransformerDecoding Strategy=Autoregressive2022.03 | 32.7 | |
| Transformer w/ LABObeam size=5, Backbone=6-layer encoder-decoder transformer2023.05 | 32.3 | |
| Transformer-base w/ BTBidirectional=false, Parameters=62M x 2, Speedup=1.0x2021.05 | 31.91 | |
| Model-level DL bigBidirectional=false, Parameters=210M x 22021.05 | 31.9 | |
| Transformer w/ LSbeam size=5, Backbone=6-layer encoder-decoder transformer2023.05 | 31.9 | |
| Transformer-base (KD teacher)Bidirectional=false, Parameters=62M x 2, Speedup=1.0x2021.05 | 31.5 | |
| MGNMTBidirectional=true, Parameters=195M2021.05 | 31.4 | |
| GLAT+CTCBidirectional=false, Parameters=62M x 2, Speedup=16.8x, Decoding=greedy2021.05 | 31.39 | |
| duplex REDER (final model)Bidirectional=true, Parameters=58M, Speedup=5.5x, Decoding=beam search (size 20)2021.05 | 31.25 | |
| GLATBidirectional=false, Parameters=62M x 2, Speedup=15.3x, Decoding=greedy2021.05 | 31.02 | |
| Transformer-base w/ MTLBidirectional=true, Parameters=62M, Speedup=1.0x2021.05 | 30.96 | |
| GLAT+CTC (our re-implementation)Bidirectional=false, Parameters=62M x 2, Speedup=16.2x, Decoding=beam search (size 20)2021.05 | 30.45 | |
| Transformer w/ CPbeam size=5, Backbone=6-layer encoder-decoder transformer2023.05 | 30.4 | |
| Transformer w/ AFLbeam size=5, Backbone=6-layer encoder-decoder transformer2023.05 | 30.3 | |
| simplex REDERBidirectional=false, Parameters=58M x 2, Speedup=5.5x, Decoding=beam search (size 20)2021.05 | 30.02 | |
| CeMATInitialization=CeMAT, Architecture=Mask-Predict, Decoding Strategy=Non-autoregressive2022.03 | 29.9 | |
| Transformerbeam size=5, Backbone=6-layer encoder-decoder transformer2023.05 | 29.8 | |
| mRASPInitialization=mRASP, Architecture=Mask-Predict, Decoding Strategy=Non-autoregressive2022.03 | 29.8 | |
| GLAT+CTC w/ MTL (our re-implementation)Bidirectional=true, Parameters=62M, Speedup=16.2x, Decoding=beam search (size 20)2021.05 | 29.49 | |
| REDER w/ MTLBidirectional=true, Parameters=58M, Speedup=5.5x, Decoding=beam search (size 20)2021.05 | 29.12 | |
| Mask-PredictDecoding Strategy=Non-autoregressive2022.03 | 29 | |
| Transformer w/ FLbeam size=5, Backbone=6-layer encoder-decoder transformer2023.05 | 28.9 | |
| KERMIT + monoBidirectional=false, Parameters=124M2021.05 | 28.6 | |
| ImputerBidirectional=false, Parameters=58M x 2, Speedup=18.6x, Decoding=greedy2021.05 | 28.4 | |
| CTCBidirectional=false, Parameters=58M x 2, Speedup=18.6x, Decoding=greedy2021.05 | 28.1 | |
| KERMITBidirectional=false, Parameters=124M2021.05 | 27.4 | |
| vanilla NATBidirectional=false, Parameters=62M x 2, Speedup=15.6x, Decoding=greedy2021.05 | 21.47 | |
| CTC w/o KDBidirectional=false, Parameters=58M x 2, Decoding=greedy2021.05 | 18.64 |