List your company
ResearchFormerly WizWand

Machine Translation on WMT En-De (newstest2014)

31.26BLEUBiBERT Contextualized Embeddings + Stochastic Layer Selection + Fine-Tuning
-0.39767.821216.0424.2588Sep 26, 2016Jul 24, 2017May 22, 2018Mar 19, 2019Jan 15, 2020Nov 11, 2020Sep 9, 2021
Updated 8mo ago

Evaluation results

MethodLinks
BiBERT Contextualized Embeddings + Stochastic Layer Selection + Fine-TuningStochastic Layer Selection=true, Fine-tuning Protocol=Fine-Tuning2021.0931.26
BiBERT Contextualized Embeddings + Stochastic Layer SelectionStochastic Layer Selection=true2021.0930.91
BERT-Fuse2021.0930.75
BERT Initialization (12 layers)Model Architecture (Layers)=12 layers2021.0930.6
BiBERT Contextualized Embeddings + Stochastic Layer Selection + Dual-Directional TrainingStochastic Layer Selection=true, Training Strategy=Dual-Directional Training2021.0930.31
Evolved Transformer2021.0929.8
Transformer + Large BatchTraining Strategy=Large Batch2021.0929.3
Transformer+ALONE (Real number)D_inter=8192, Embed Params=8.4M2020.0427.61
Transformer+ALONE (Binary)D_inter=8192, Embed Params=8.4M2020.0427.55
TransformerEmbed Params=16.8M2020.0427.3
Transformer (conventional word embeddings)Embed Params=16.8M2020.0427.12
Transformer+DeFINE2020.0427.01
Transformer+ALONE (Binary)D_inter=4096, Embed Params=4.2M2020.0426.97
Transformer+ALONE (Real number)D_inter=8192, Embed Params=8.4M, Trainable Sigma=false2020.0426.95
Transformer+ALONE (Real number)D_inter=4096, Embed Params=4.2M2020.0426.93
Transformer+ALONE (Binary)D_inter=8192, Embed Params=8.4M, Trainable Sigma=false2020.0426.9
Transformer+ALONE (Real number)D_inter=4096, Embed Params=4.2M, Trainable Sigma=false2020.0426.85
Transformer+ALONE (Binary)D_inter=4096, Embed Params=4.2M, Trainable Sigma=false2020.0426.75
Transformer (factorized embed)Embed Params=8.5M2020.0426.56
Transformer (factorized embed)Embed Params=4.3M2020.0426.43
RL-refined WPM-32Kensemble_size=8 models, vocabulary=32K wordpieces, RL_refinement=true2016.0926.3
WPM-32Kensemble_size=8 models, vocabulary=32K wordpieces2016.0926.2
ConvS2SEmbed Params=66.0M2020.0425.2
GNMT (RNN Enc-Dec Att)Inputs=word-pieces, Outputs=word-pieces2016.1024.61
GNMTSupervision=Supervised2017.1024.61
ByteNetInputs=char, Outputs=char2016.1023.75
GNMT (RNN Enc-Dec Att)Inputs=char, Outputs=char2016.1022.62
RNN Enc-Dec Att (Chung et al.)Inputs=BPE, Outputs=char2016.1021.33
RNN Enc-Dec Att (Luong et al.)Inputs=words, Outputs=words2016.1020.9
Phrase Based MTInputs=phrases, Outputs=phrases2016.1020.7
RNN Enc-Dec Att (Zhou et al.)Inputs=words, Outputs=words2016.1020.6
RNN Enc-Dec Att (Chung et al.)Inputs=BPE, Outputs=BPE2016.1019.98
Reverse RNN Enc-DecInputs=words, Outputs=words2016.1014
RNN Enc-DecInputs=words, Outputs=words2016.1011.3
Comparable NMT (full parallel)Supervision=Supervised, Parallel data=full2017.1011.05
Proposed (full) + 100k parallelSupervision=Semi-supervised, Parallel data=100k2017.1010.95
Proposed (full) + 10k parallelSupervision=Semi-supervised, Parallel data=10k2017.107.86
Proposed (+ BPE)Supervision=Unsupervised, Components=denoising, backtranslation, BPE2017.106.89
Proposed (+ backtranslation)Supervision=Unsupervised, Components=denoising, backtranslation2017.106.55
Comparable NMT (100k parallel)Supervision=Supervised, Parallel data=100k2017.105.29
Baseline (emb. nearest neighbor)Supervision=Unsupervised2017.104.39
Proposed (denoising)Supervision=Unsupervised, Components=denoising2017.102.4
Comparable NMT (10k parallel)Supervision=Supervised, Parallel data=10k2017.100.82