# Multi-modality Latent Interaction Network for Visual Question Answering

Source: [Multi-modality Latent Interaction Network for Visual Question Answering](<https://www.sota2.com/research/paper/68fa663b1e209131cea0f06a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:45:30.461Z

Authors: Peng Gao, Haoxuan You, Zhanpeng Zhang, Xiaogang Wang, Hongsheng Li

Published: 2019-08-10T05:57:01.000Z

## Abstract

Exploiting relationships between visual regions and question words have achieved great success in learning multi-modality features for Visual Question Answering (VQA). However, we argue that existing methods mostly model relations between individual visual regions and words, which are not enough to correctly answer the question. From humans' perspective, answering a visual question requires understanding the summarizations of visual and language information. In this paper, we proposed the Multi-modality Latent Interaction module (MLI) to tackle this problem. The proposed module learns the cross-modality relationships between latent visual and language summarizations, which summarize visual regions and question into a small number of latent representations to avoid modeling uninformative individual region-word relations. The cross-modality information between the latent summarizations are propagated to fuse valuable information from both modalities and are used to update the visual and word features. Such MLI modules can be stacked for several stages to model complex and latent relations between the two modalities and achieves highly competitive performance on public VQA benchmarks, VQA v2.0 and TDIUC . In addition, we show that the performance of our methods could be significantly improved by combining with pre-trained language model BERT.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Visual Question Answering on VQA v2 (test-dev)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-test-dev.md>): Visual Question Answering; VQA v2 (test-dev); Overall Accuracy: 71.09.

- [Visual Question Answering on VQA v2 (test-std)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-test-std.md>): Visual Question Answering; VQA v2 (test-std); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1908.04289>)

[PDF](<https://arxiv.org/pdf/1908.04289.pdf>)