List your company
ResearchFormerly WizWand

Machine Translation on WMT De-En 14 (test)

34.19BLEUBPE-dropout
10.093216.349122.60528.8609Nov 7, 2017Aug 12, 2018May 18, 2019Feb 21, 2020Nov 26, 2020Sep 1, 2021Jun 7, 2022
Updated 8mo ago

Evaluation results

MethodLinks
BPE-dropoutSegmentation=BPE-dropout2019.1034.19——
Kudo (2018)Segmentation=Unigram Language Model2019.1033.65——
BPESegmentation=BPE2019.1032.69——
LATIterations=4, latency (ms)=732020.1132.04——
ATIterations=N, latency (ms)=4862020.1131.87——
FullyNAT + CTC + GLATCategory=Alignment-Based, Inference mode=single-step2022.0631.4——
DATCategory=Alignment-Based, Inference mode=single-step2022.0631.3——
CMLMC + DiMSCategory=Proposed, Inference mode=single-step2022.0631.1——
JM-NATIdec=102020.0831.02—5.7
GLAT + NPDIdec=1, m=72020.0831.02—7.9
Flowseq + NPDIdec=1, m=302020.0830.68——
CMLMIterations=10, latency (ms)=1662020.1130.53——
Mask-PredictIdec=10, type=Iterative NAT2020.0830.53—1.7
AlignNARTCategory=Alignment-Based, Inference mode=single-step2022.0630.4——
OAXECategory=Alignment-Based, Inference mode=single-step2022.0630.2——
LATIterations=1, latency (ms)=312020.1129.91——
CMLMIterations=4, latency (ms)=722020.1129.9——
GLATIdec=12020.0829.84—15.3
GLATCategory=XE-Based, Inference mode=single-step2022.0629.8——
NAT-DCRF + NPDIdec=1, m=92020.0829.68—6.1
GLAT + CTCIdec=12020.0829.54—14.6
NAT-HINT + NPDIdec=1, m=92020.0829.52——
CTC*Idec=12020.0828.73—14.6
ImputerIdec=12020.0828.4—18.6
ImputerCategory=Alignment-Based, Inference mode=single-step2022.0628.4——
FlowseqIdec=12020.0828.39—1.1
ENGINECategory=XE-Based, Inference mode=single-step2022.0628.1——
EM+ODDCategory=XE-Based, Inference mode=single-step2022.0627.9——
AXECategory=Alignment-Based, Inference mode=single-step2022.0627.9——
Transformerimplementation=ours2020.0827.48—1
imit-NAT + NPDIdec=1, m=72020.0827.28—9.7
NAT-DCRFIdec=12020.0827.22—10.4
Bigram CRFCategory=XE-Based, Inference mode=single-step2022.0627.2——
Autoregressivebeam size=42017.1127.026071
Autoregressivebeam size=12017.1126.394081.49
CNATDecoding=argmax decoding2021.0325.73——
imit-NATIdec=12020.0825.67—18.6
SynSTDecoding=argmax decoding2021.0325.5——
NAT-IRIdec=102020.0825.48—1.5
FlowseqDecoding=argmax decoding2021.0325.4——
NAT-HINTIdec=12020.0825.24——
Hint-based LossCategory=XE-Based, Inference mode=single-step2022.0625.2——
AXE CMLMDecoding=argmax decoding2021.0324.9——
NAT-base*Idec=12020.0824.81—15.3
Aux. Reg.Category=XE-Based, Inference mode=single-step2022.0624.8——
Bag-of-ngramCategory=XE-Based, Inference mode=single-step2022.0624.6——
SMARTCategory=XE-Based, Inference mode=single-step2022.0623.8——
CMLMCCategory=XE-Based, Inference mode=single-step2022.0623.6——
NATdecoding=Noisy Parallel Decoding, fine-tuning=true, sample size=1002017.1123.22572.36
NAT-FT + NPDIdec=1, m=1002020.0823.2—2.4
NATdecoding=Noisy Parallel Decoding, fine-tuning=true, sample size=102017.1122.41797.68
CMLMIterations=1, latency (ms)=272020.1121.83——
Mask-PredictIdec=1, type=Fully NAT2020.0821.83——
CMLMCategory=XE-Based, Inference mode=single-step2022.0621.8——
NATdecoding=argmax, fine-tuning=true2017.1121.473915.6
NAT-FTIdec=12020.0821.47—15.6
NATdecoding=argmax2017.1120.623915.6
NAT-CTCIdec=12020.0818.64——
NAT (ours)Decoding=argmax decoding2021.0311.02——