# Video-guided Machine Translation with Global Video Context

Source: [Video-guided Machine Translation with Global Video Context](<https://www.sota2.com/research/paper/69d732b77d0172140ec76ce6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-10T05:00:04.898Z

Authors: Jian Chen, JinZe Lv, Zi Long, XiangHua Fu

Published: 2026-04-08T07:57:05.000Z

## Abstract

Video-guided Multimodal Translation (VMT) has advanced significantly in recent years. However, most existing methods rely on locally aligned video segments paired one-to-one with subtitles, limiting their ability to capture global narrative context across multiple segments in long videos. To overcome this limitation, we propose a globally video-guided multimodal translation framework that leverages a pretrained semantic encoder and vector database-based subtitle retrieval to construct a context set of video segments closely related to the target subtitle semantics. An attention mechanism is employed to focus on highly relevant visual content, while preserving the remaining video features to retain broader contextual information. Furthermore, we design a region-aware cross-modal attention mechanism to enhance semantic alignment during translation. Experiments on a large-scale documentary translation dataset demonstrate that our method significantly outperforms baseline models, highlighting its effectiveness in long-video scenarios.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multimodal Machine Translation on TopicVD (test)](<https://www.sota2.com/research/sota/multimodal-machine-translation-on-topicvd-test.md>): Multimodal Machine Translation; TopicVD (test); BLEU: 30.47.

- [Video-guided Multimodal Translation on BigVideo (subset)](<https://www.sota2.com/research/sota/video-guided-multimodal-translation-on-bigvideo-subset.md>): Video-guided Multimodal Translation; BigVideo (subset); BLEU: 46.78.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.06789>)

[PDF](<https://arxiv.org/pdf/2604.06789.pdf>)
