# Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation

Source: [Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation](<https://www.sota2.com/research/paper/68fa646a1e209131cea04963>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:04:03.574Z

Authors: Jean-Benoit Delbrouck, St\\'ephane Dupont, Omar Seddati

Published: 2017-07-04T14:26:56.000Z

## Abstract

In Multimodal Neural Machine Translation (MNMT), a neural model generates a translated sentence that describes an image, given the image itself and one source descriptions in English. This is considered as the multimodal image caption translation task. The images are processed with Convolutional Neural Network (CNN) to extract visual features exploitable by the translation model. So far, the CNNs used are pre-trained on object detection and localization task. We hypothesize that richer architecture, such as dense captioning models, may be more suitable for MNMT and could lead to improved translations. We extend this intuition to the word-embeddings, where we compute both linguistic and visual representation for our corpus vocabulary. We combine and compare different confi

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1707.01009>)

[PDF](<https://arxiv.org/pdf/1707.01009.pdf>)
