List your company
ResearchFormerly WizWand

Machine Translation on IWSLT En-De 2014 (test)

37.85BLEUBaseline + relaxed cross attention
19.670824.390429.1133.8296Aug 11, 2018Jul 31, 2019Jul 19, 2020Jul 8, 2021Jun 27, 2022Jun 16, 2023Jun 4, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Baseline + relaxed cross attentionLanguage Model=MuST-C additional data, matched inference=false2022.0937.85
Baseline + relaxed self-attentionLanguage Model=MuST-C additional data, matched inference=false2022.0937.74
Baseline + relaxed self-attentionLanguage Model=MuST-C additional data, matched inference=true2022.0937.71
Baseline + smoothed focusLanguage Model=MuST-C additional data2022.0937.67
Baseline + relaxed self-attentionLanguage Model=None, matched inference=true2022.0937.67
Baseline + relaxed self-attentionLanguage Model=IWSLT14 training transcripts, matched inference=true2022.0937.67
Baseline + relaxed cross attentionLanguage Model=IWSLT14 training transcripts, matched inference=true2022.0937.64
Baseline (Shen et al. resimulated)Language Model=MuST-C additional data2022.0937.62
Shen et al.Language Model=None2022.0937.6
Baseline + relaxed cross attentionLanguage Model=None, matched inference=true2022.0937.6
Baseline + relaxed cross attentionLanguage Model=MuST-C additional data, matched inference=true2022.0937.57
Baseline + relaxed self-attentionLanguage Model=None, matched inference=false2022.0937.57
Baseline + relaxed cross attentionLanguage Model=None, matched inference=false2022.0937.56
Baseline + relaxed cross attentionLanguage Model=IWSLT14 training transcripts, matched inference=false2022.0937.53
Baseline + smoothed focusLanguage Model=IWSLT14 training transcripts2022.0937.52
Baseline + relaxed self-attentionLanguage Model=IWSLT14 training transcripts, matched inference=false2022.0937.49
Baseline (Shen et al. resimulated)Language Model=None2022.0937.42
Baseline (Shen et al. resimulated)Language Model=IWSLT14 training transcripts2022.0937.42
Baseline + smoothed focusLanguage Model=None2022.0937.42
Liang et al.Language Model=None2022.0937.25
Wu et al. [59]Language Model=None2022.0936.88
AdminRemove LN=false, w_skip=true, Warmup=false, Optimizer=Adam2021.0236.1
GradInitRemove LN=false, w_skip=true, Warmup=false, Optimizer=Adam2021.0236.1
GradInitRemove LN=false, w_skip=false, Warmup=false, Optimizer=Adam2021.0236
AdminRemove LN=false, w_skip=true, Warmup=true, Optimizer=RAdam2021.0235.7
StandardRemove LN=false, w_skip=false, Warmup=true, Optimizer=RAdam2021.0235.6
GradInitRemove LN=false, w_skip=true, Warmup=false, Optimizer=SGD2021.0235.6
T-FixUpRemove LN=true, w_skip=true, Warmup=false, Optimizer=Adam2021.0235.5
Wu et al. [58]Language Model=None2022.0935.2
FixUpRemove LN=true, w_skip=false, Warmup=false, Optimizer=Adam2021.0234.5
Fan et al.Language Model=None2022.0934.5
Vaswani et al.Language Model=None2022.0934.4
AdminRemove LN=false, w_skip=true, Warmup=false, Optimizer=SGD2021.0233.7
Bi-SimCutbidirectional pretraining=true2022.0631.16
Document-level + BERTContext level=document-level, BERT-fused=true2020.0231.02
DRDAgranularity=dynamic2024.0630.92
DRDAgranularity=static2024.0630.84
R-Drop2022.0630.73
CipherDAug - 2 keyssrc aug=false, tgt aug=false, |Θ|=3x2022.0430.65
Data Diversesrc aug=true, tgt aug=true, |Θ|=7x2022.0430.47
Sentence-level + BERTContext level=sentence-level, BERT-fused=true2020.0230.45
BiBERTbidirectional pretraining=true2022.0630.45
UniDropbase_model=Transformer2021.0429.99
UniDrop2022.0629.99
MixReps+co-teaching2021.0429.93
Mixed Rep2022.0629.93
MAT2021.0429.9
Mixed-Repr.src aug=true, tgt aug=true, |Θ|=2x2022.0429.71
VAT2022.0629.45
NPRF enc-dec w/ RPEComplexity (encoder/decoder)=O(nlogn)/O(nlogn), Relative Positional Encoding=true2021.0629.4
Standard enc + PRF decComplexity (encoder/decoder)=O(n²)/O(n), Encoder Attention=softmax, Decoder Attention=PRF2021.0629.3
WordDropoutsrc aug=true, tgt aug=false, |Θ|=1x2022.0429.21
Standard enc-decComplexity (encoder/decoder)=O(n²)/O(n²), Encoder Attention=softmax, Decoder Attention=softmax2021.0629.2
CipherDAug - 1 keysrc aug=false, tgt aug=false, |Θ|=2x2022.0429.14
RAML+SwitchOutsrc aug=true, tgt aug=true, |Θ|=1x2022.0429.11
RAMLsrc aug=false, tgt aug=true, |Θ|=1x2022.0429.07
Transformer2024.0629.03
SwitchOutsrc aug=true, tgt aug=false, |Θ|=1x2022.0429
Our Document-levelContext level=document-level, BERT-fused=false2020.0228.9
Transformer-base+Self-ExplainingArchitecture=Transformer-base, Self-Explaining=true2020.1228.9
Transformer w/ LABObeam size=5, Backbone=6-layer encoder-decoder transformer2023.0528.8
RAML+WordDropoutsrc aug=true, tgt aug=true, |Θ|=1x2022.0428.78
Transformer2022.0628.7
Transformer2021.0428.67
BPE-Dropoutsrc aug=true, tgt aug=true, |Θ|=1x2022.0428.63
Sentence-level baselineContext level=sentence-level, BERT-fused=false2020.0228.57
Transformersrc aug=false, tgt aug=false, |Θ|=1x2022.0428.57
Transformer-baseArchitecture=Transformer-base2020.1228.4
Transformer w/ LSbeam size=5, Backbone=6-layer encoder-decoder transformer2023.0528.3
Transformer**Tokenization=BPE-based, Version=V1, Implementation=FairSeq, Number of parameters=48M2018.0828.23
Transformer**Tokenization=BPE-based, Version=V2, Implementation=FairSeq, Number of parameters=52M2018.0828.07
Pervasive AttentionTokenization=BPE-based, Version=V2, Number of parameters=22M2018.0827.99
Miculicich et al. (2018)Context level=document-level, BERT-fused=false2020.0227.94
Transformer w/ CPbeam size=5, Backbone=6-layer encoder-decoder transformer2023.0527.9
Transformer w/ AFLbeam size=5, Backbone=6-layer encoder-decoder transformer2023.0527.9
Transformerbeam size=5, Backbone=6-layer encoder-decoder transformer2023.0527.8
PRF enc-decComplexity (encoder/decoder)=O(n)/O(n), Encoder Attention=PRF, Decoder Attention=PRF2021.0627.6
Pervasive AttentionTokenization=BPE-based, Version=V1, Number of parameters=11M2018.0827.21
Transformer w/ FLbeam size=5, Backbone=6-layer encoder-decoder transformer2023.0526.8
ConvS2S** (MLE)Tokenization=BPE-based, Version=V1, Implementation=FairSeq, Number of parameters=22M2018.0826.73
CeMATInitialization=CeMAT, Architecture=Mask-Predict, Decoding Strategy=Non-autoregressive2022.0326.7
NPMT + language modelTokenization=Word-based2018.0825.36
RNNsearch*Tokenization=BPE-based, Version=V1, Implementation=our implementation, Number of parameters=15M2018.0825.04
RIMsTraining Data=en -> de, en -> fr2019.0924.23
TransformerDecoding Strategy=Autoregressive2022.0323.9
mRASPInitialization=mRASP, Architecture=Mask-Predict, Decoding Strategy=Non-autoregressive2022.0323.9
RIMsTraining Data=en -> de2019.0923.71
TransformerTraining Data=en -> de, en -> fr2019.0922.92
TransformerTraining Data=en -> de2019.0922.89
Mask-PredictDecoding Strategy=Non-autoregressive2022.0322
LSTMTraining Data=en -> de2019.0921.32
LSTMTraining Data=en -> de, en -> fr2019.0920.37