Multimodal Machine Translation on Multi30k En-De 2017 (test)
62.8METEORVALHALLA (M)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VALHALLA (M)Translation Setting=Multimodal, Ground-truth Visual Tokens usage=true, Model Averaging=true2022.05 | 62.8 | 35.1 | |
| VALHALLATranslation Setting=Text-Only, Ground-truth Visual Tokens usage=false, Model Averaging=true2022.05 | 62.8 | 35.1 | |
| VALHALLA (M)Translation Setting=Multimodal, Ground-truth Visual Tokens usage=true, Model Averaging=false2022.05 | 62.5 | 34 | |
| VALHALLATranslation Setting=Text-Only, Ground-truth Visual Tokens usage=false, Model Averaging=false2022.05 | 62.5 | 34 | |
| Selective AttnFeature=ViT-Large2022.03 | 62.32 | 34.32 | |
| Selective Attn + QueryInstFeature=QueryInst2022.03 | 62.27 | 34.9 | |
| Transformer-Tiny#Params=2.6M2021.05 | 62.05 | — | |
| Transformer TinyFeature=Text-only2022.03 | 62.05 | 33.36 | |
| Gated Fusion#Params=2.9M, configuration=Tiny2021.05 | 61.94 | — | |
| Gated FusionFeature=ResNet2022.03 | 61.94 | 33.59 | |
| Gated FusionTranslation Setting=Multimodal2022.05 | 61.9 | 33.6 | |
| Doubly-ATT#Params=3.2M, configuration=Tiny2021.05 | 61.83 | — | |
| Doubly-ATTFeature=ResNet2022.03 | 61.83 | 33.95 | |
| Selective Attn + CATRFeature=CATR2022.03 | 61.81 | 34.28 | |
| RMMT#Params=2.9M, configuration=Tiny2021.05 | 61.71 | — | |
| RMMTTranslation Setting=Text-Only2022.05 | 61.7 | 32.9 | |
| Gated FusionFeature=ViT-Large2022.03 | 61.67 | 33.49 | |
| Selective Attn + ViT-SmallFeature=ViT-Small2022.03 | 61.61 | 33.62 | |
| Selective Attn + DETRFeature=DETR2022.03 | 61.57 | 34.14 | |
| Selective Attn + ViT-BaseFeature=ViT-Base2022.03 | 61.42 | 33.6 | |
| Imagination#Params=7.0M, configuration=Tiny2021.05 | 61.29 | — | |
| ImaginationFeature=ResNet2022.03 | 61.29 | 32.89 | |
| Transformer-Small#Params=36.5M2021.05 | 60.8 | — | |
| Selective Attn + ViT-TinyFeature=ViT-Tiny2022.03 | 60.46 | 32.48 | |
| Transformer-Base#Params=49.1M2021.05 | 60.02 | — | |
| IKD-MMTSystem Category=Image-free2022.10 | 53.21 | 33.83 | |
| DS-SUM-L2System Category=Image-must2022.10 | 52.9 | 32.6 | |
| ImagiTTranslation Setting=Text-Only2022.05 | 52.4 | 32.1 | |
| OVC+LmSystem Category=Image-must2022.10 | 52.4 | 32.3 | |
| ImagiTSystem Category=Image-free2022.10 | 52.4 | 32.1 | |
| CAP-ALLTranslation Setting=Multimodal2022.05 | 52.2 | 33 | |
| Trg-mulSystem Category=Image-must2022.10 | 52.2 | 30.7 | |
| VAG-NMTSystem Category=Image-must2022.10 | 52.2 | 31.6 | |
| TransformerSystem Category=Image-free2022.10 | 52.1 | 31.7 | |
| GMNMT#Params=4.0M2021.05 | 51.9 | — | |
| GMNMTTranslation Setting=Multimodal2022.05 | 51.9 | 32.2 | |
| GMNMTSystem Category=Image-must2022.10 | 51.9 | 32.2 | |
| Gumbel-AttentionTranslation Setting=Multimodal2022.05 | 51.2 | 31.4 | |
| Fusion-convSystem Category=Image-must2022.10 | 51.2 | 29.8 | |
| Gumbel-attSystem Category=Image-must2022.10 | 51.2 | 31.4 | |
| DCCN#Params=17.1M2021.05 | 49.9 | — | |
| DCCNTranslation Setting=Multimodal2022.05 | 49.9 | 31 | |
| VMMTFTranslation Setting=Text-Only2022.05 | 49.9 | 30.1 | |
| DCCNSystem Category=Image-must2022.10 | 49.9 | 31 | |
| VMMTFSystem Category=Image-free2022.10 | 49.9 | 30.1 | |
| Gated FusionSystem Category=Image-must2022.10 | — | 33.59 | |
| RMMTSystem Category=Image-must2022.10 | — | 32.94 | |
| UVR-NMTFeature=ResNet2022.03 | — | 32.16 | |
| UVR-NMTTranslation Setting=Text-Only2022.05 | — | 28.6 | |
| UVR-NMTSystem Category=Image-free2022.10 | — | 28.63 |