# Multimodal Integration of Human-Like Attention in Visual Question Answering Source: [Multimodal Integration of Human-Like Attention in Visual Question Answering]() Collection index: [llms.txt]() Data updated: 2026-03-05T05:00:11.275Z Authors: Ekta Sood, Fabian K\\"ogel, Philipp M\\"uller, Dominike Thomas, Mihai Bace, Andreas Bulling Published: 2021-09-27T15:56:54.000Z ## Abstract Human-like attention as a supervisory signal to guide neural attention has shown significant promise but is currently limited to uni-modal integration - even for inherently multimodal tasks such as visual question answering (VQA). We present the Multimodal Human-like Attention Network (MULAN) - the first method for multimodal integration of human-like attention on image and text during training of VQA models. MULAN integrates attention predictions from two state-of-the-art text and image saliency models into neural self-attention layers of a recent transformer-based VQA model. Through evaluations on the challenging VQAv2 dataset, we show that MULAN achieves a new state-of-the-art performance of 73.98% accuracy on test-std and 73.72% on test-dev and, at the same time, has approximately 80% fewer trainable parameters than prior work. Overall, our work underlines the potential of integrating multimodal human-like and neural attention for VQA ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Visual Question Answering on VQA v2 (test-std)](): Visual Question Answering; VQA v2 (test-std); Accuracy: 73.98. - [Visual Question Answering on VQA v2 (test-dev)](): Visual Question Answering; VQA v2 (test-dev); Overall Accuracy: 73.72. ## Original sources and implementation [Original paper]() [PDF]()