Machine Translation (Multi-Pair Evaluation) on TED (test)
28.3RU-EN ScoreSoftmax Model (Decoupled) - supervised fine-tune
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Softmax Model (Decoupled) - supervised fine-tuneModel Type=Softmax, L_base=5, d_model=3002024.08 | 28.3 | — | — | 27.4 | 45.2 | 37.8 | 39.6 | 33.5 | 27.7 | 19.3 | — | |
| Softmax Model (Decoupled)Model Type=Softmax, Trained on=en, de, es, fr, it, ru, uk, pl, vi, zh, L_base=10, d_model=5762024.08 | 28 | 37.1 | 43.1 | 28.7 | 41.6 | 31.2 | 33 | 21.5 | 11.9 | 7.7 | — | |
| Garcia et al. (2021)Model Type=Softmax, L_base=5, d_model=3002024.08 | 27.4 | — | — | 29.7 | 44 | 38.6 | 40.9 | 32 | 27.7 | 21.4 | — | |
| vMF ModelModel Type=vMF, Trained on=en, de, es, fr, it, ru, uk, pl, L_base=8, d_model=3002024.08 | 25.9 | 34.5 | 40 | 12.6 | 37.8 | 28.6 | 30.5 | 16.3 | 10.1 | 6.3 | — | |
| vMF Model - supervised fine-tuneModel Type=vMF, L_base=5, d_model=3002024.08 | 25.7 | 36.1 | 42 | 25.3 | 60.4 | 35.4 | 38.1 | 29 | — | 15 | — | |
| vMF ModelModel Type=vMF, Trained on=en, de, es, fr, it, ru, uk, pl, vi, zh, L_base=10, d_model=3002024.08 | 25.3 | 33.6 | 39.2 | 26.4 | 36.5 | 28 | 29.7 | 16.5 | 10.1 | 6.6 | — | |
| Softmax Model (Decoupled)Model Type=Softmax, Trained on=en, de, es, fr, it, L_base=5, d_model=3002024.08 | 20.7 | 38.8 | 44.5 | 13 | 42.6 | 31.5 | 29 | 17.6 | 9.5 | 5.6 | — | |
| vMF ModelModel Type=vMF, Trained on=en, de, es, fr, it, L_base=5, d_model=3002024.08 | 18.9 | 36 | 41.7 | 10.2 | 39.6 | 28.3 | 24 | 14.5 | 8.3 | 4.5 | — | |
| vMF ModelModel Type=vMF, Trained on=en, de, fr, it, L_base=4, d_model=3002024.08 | 18.3 | 36 | 35.8 | 10 | 36.9 | 28 | 24.3 | 13.3 | 7.8 | 4.5 | — | |
| vMF ModelModel Type=vMF, Trained on=es-en, L_base=2, d_model=3002024.08 | 15.6 | 16.8 | 41.8 | 7.8 | 35.3 | 20.4 | 18 | 11.9 | 7.2 | 4.2 | — | |
| vMF ModelModel Type=vMF, Trained on=es-en, L_base=1, d_model=3002024.08 | 12.8 | 11.4 | 42.4 | 5.7 | 34.2 | 16.6 | 12 | 8.8 | 5.5 | 3.4 | — | |
| BPE baselineModel Type=Softmax, L_base=5, d_model=3002024.08 | 2.1 | 39 | 44.4 | 2 | 12.9 | 5.6 | 2 | 2.7 | 2.9 | 3.5 | — | |
| BPE baselineArchitecture=Softmax, L_base=5, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 11.1 | |
| Multilingual SoftmaxArchitecture=Softmax, Trained on=en, de, es, fr, it, L_base=5, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 25.1 | |
| Multilingual SoftmaxArchitecture=Softmax, Trained on=en, de, es, fr, it, ru, uk, pl, vi, zh, L_base=10, d_model=5762024.08 | — | — | — | — | — | — | — | — | — | — | 25 | |
| Supervised fine-tuneArchitecture=vMF, L_base=5, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 33.4 | |
| vMF-NMTArchitecture=vMF, Trained on=es, en, L_base=1, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 21.6 | |
| vMF-NMTArchitecture=vMF, Trained on=es+en, L_base=2, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 22.6 | |
| vMF-NMTArchitecture=vMF, Trained on=en, de, fr, it, L_base=4, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 21.4 | |
| vMF-NMTArchitecture=vMF, Trained on=en, de, es, fr, it, L_base=5, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 20.4 | |
| vMF-NMTArchitecture=vMF, Trained on=en, de, es, fr, it, ru, uk, pl, L_base=8, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 22.3 | |
| vMF-NMTArchitecture=vMF, Trained on=en, de, es, fr, it, ru, uk, pl, vi, zh, L_base=10, d_model=3002024.08 | — | — | — | — | — | — | — | — | — | — | 23 |