List your company
ResearchFormerly WizWand

Machine Translation on WMT Ro-En 2016 (test)

37.8BLEUmBART
17.5222.78528.0533.315Nov 7, 2017Apr 1, 2019Aug 24, 2020Jan 17, 2022Jun 12, 2023Nov 4, 2024Mar 30, 2026
Updated 7mo ago

Evaluation results

MethodLinks
mBARTTraining Size=597K, Pre-trained=true2022.0337.8——
CeMATTraining Size=597K, Pre-trained=true2022.0337.1——
mRASPTraining Size=597K, Pre-trained=true2022.0336.9——
XLMTraining Size=597K, Pre-trained=true2022.0335.6——
Topdown2026.0334.17——
GLAT+CTCBidirectional=false, Parameters=62M x 2, Speedup=16.8x, Decoding=greedy2021.0534.16——
Transformerimplementation=ours2020.0834.05—1
duplex REDER (final model)Bidirectional=true, Parameters=58M, Speedup=5.5x, Decoding=beam search (size 20)2021.0534.03——
DirectTraining Size=597K, Pre-trained=false2022.0334——
ATIterations=N, latency (ms)=4862020.1133.98——
Transformer2026.0333.93——
MGNMTBidirectional=true, Parameters=195M2021.0533.9——
GLAT + CTCIdec=12020.0833.84—14.6
Transformer-base (KD teacher)Bidirectional=false, Parameters=62M x 2, Speedup=1.0x2021.0533.7——
GLAT + NPDIdec=1, m=72020.0833.51—7.9
GLATBidirectional=false, Parameters=62M x 2, Speedup=15.3x, Decoding=greedy2021.0533.51——
CTC*Idec=12020.0833.46—14.6
CMLMIterations=10, latency (ms)=1662020.1133.31——
Mask-PredictIdec=10, type=Iterative NAT2020.0833.31—1.7
LATIterations=4, latency (ms)=732020.1133.26——
LevTIdec=6+2020.0833.26—4
CMLMIterations=4, latency (ms)=722020.1133.23——
1.5-entmaxactivation=1.5-entmax2019.0833.1——
simplex REDERBidirectional=false, Parameters=58M x 2, Speedup=5.5x, Decoding=beam search (size 20)2021.0532.98——
alpha-entmaxactivation=alpha-entmax2019.0832.89——
Flowseq + NPDIdec=1, m=302020.0832.84——
softmaxactivation=softmax2019.0832.7——
GLATIdec=12020.0832.04—15.3
imit-NAT + NPDIdec=1, m=72020.0831.81—9.7
XLMEncoder Pre-training=MLM, Decoder Pre-training=MLM2019.0131.8——
Autoregressivebeam size=42017.1131.766071
ImputerIdec=12020.0831.7—18.6
ImputerBidirectional=false, Parameters=58M x 2, Speedup=18.6x, Decoding=greedy2021.0531.7——
CTCBidirectional=false, Parameters=58M x 2, Speedup=18.6x, Decoding=greedy2021.0531.6——
NATdecoding=Noisy Parallel Decoding, fine-tuning=true, sample size=1002017.1131.442572.36
NAT-FT + NPDIdec=1, m=1002020.0831.44—2.4
LATIterations=1, latency (ms)=312020.1131.24——
Autoregressivebeam size=12017.1131.034081.49
CONTBackbone=T5-small, Loss Function=N-Pairs loss, target-source representation similarity=true2022.0530.91——
NATdecoding=Noisy Parallel Decoding, fine-tuning=true, sample size=102017.1130.76797.68
FlowseqIdec=12020.0830.72—1.1
CONTBackbone=T5-small, Loss Function=N-Pairs loss, target-source representation similarity=false2022.0530.54——
XLMEncoder Pre-training=MLM, Decoder Pre-training=-2019.0130.5——
XLMEncoder Pre-training=MLM, Decoder Pre-training=CLM2019.0130.4——
NAT-IRIdec=102020.0830.19—1.5
Naive CLBackbone=T5-small, Loss Function=N-Pairs loss, target-source representation similarity=true2022.0529.86——
XLMEncoder Pre-training=CLM, Decoder Pre-training=MLM2019.0129.8——
Naive CLBackbone=T5-small, Loss Function=N-Pairs loss, target-source representation similarity=false2022.0529.74——
CONTBackbone=T5-small, Loss Function=InfoNCE loss2022.0529.64——
NAT-base*Idec=12020.0829.43—15.3
CLAPSBackbone=T5-small, Loss Function=InfoNCE loss2022.0529.41——
LaNMTIdec=42020.0829.1—5.7
Dropout CLBackbone=T5-small, Loss Function=InfoNCE loss2022.0529.1——
NATdecoding=argmax, fine-tuning=true2017.1129.063915.6
NAT-FTIdec=12020.0829.06—15.6
vanilla NATBidirectional=false, Parameters=62M x 2, Speedup=15.6x, Decoding=greedy2021.0529.06——
imit-NATIdec=12020.0828.9—18.6
SSMBA CLBackbone=T5-small, Loss Function=InfoNCE loss2022.0528.48——
MLEBackbone=T5-small, Loss Function=MLE2022.0528.21——
CMLMIterations=1, latency (ms)=272020.1128.2——
Mask-PredictIdec=1, type=Fully NAT2020.0828.2——
XLMEncoder Pre-training=-, Decoder Pre-training=CLM2019.0128——
NATdecoding=argmax2017.1127.833915.6
XLMEncoder Pre-training=CLM, Decoder Pre-training=CLM2019.0127.8——
Naive CLBackbone=T5-small, Loss Function=InfoNCE loss2022.0527.79——
CONTBackbone=Transformer-small, Loss Function=N-Pairs loss, target-source representation similarity=true2022.0527.7——
CONTBackbone=Transformer-small, Loss Function=N-Pairs loss, target-source representation similarity=false2022.0527.42——
XLMEncoder Pre-training=MLM, Decoder Pre-training=-2019.0127.3——
XLMEncoder Pre-training=EMB, Decoder Pre-training=EMB2019.0126.6——
Naive CLBackbone=Transformer-small, Loss Function=N-Pairs loss, target-source representation similarity=false2022.0526.27——
Naive CLBackbone=Transformer-small, Loss Function=N-Pairs loss, target-source representation similarity=true2022.0526.15——
Dropout CLBackbone=Transformer-small, Loss Function=InfoNCE loss2022.0526.01——
SSMBA CLBackbone=Transformer-small, Loss Function=InfoNCE loss2022.0525.98——
MLEBackbone=Transformer-small, Loss Function=MLE2022.0525.78——
CONTBackbone=Transformer-small, Loss Function=InfoNCE loss2022.0525.74——
Naive CLBackbone=Transformer-small, Loss Function=InfoNCE loss2022.0525.49——
NAT-CTCIdec=12020.0824.67——
CTC w/o KDBidirectional=false, Parameters=58M x 2, Decoding=greedy2021.0524.67——
XLMEncoder Pre-training=CLM, Decoder Pre-training=-2019.0124.6——
PBSMT + NMT2019.0123.9——
CLAPSBackbone=Transformer-small, Loss Function=InfoNCE loss2022.0523.59——
PBSMT2019.0123——
NMT2019.0119.4——
XLMEncoder Pre-training=-, Decoder Pre-training=-2019.0118.3——