# Localized Questions in Medical Visual Question Answering

Source: [Localized Questions in Medical Visual Question Answering](<https://www.sota2.com/research/paper/68fa6fe71e209131cea480bc>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:34:50.648Z

Authors: Sergio Tascon-Morales, Pablo M\\'arquez-Neila, Raphael Sznitman

Published: 2023-07-03T14:47:18.000Z

## Abstract

Visual Question Answering (VQA) models aim to answer natural language questions about given images. Due to its ability to ask questions that differ from those used when training the model, medical VQA has received substantial attention in recent years. However, existing medical VQA models typically focus on answering questions that refer to an entire image rather than where the relevant content may be located in the image. Consequently, VQA models are limited in their interpretability power and the possibility to probe the model about specific image regions. This paper proposes a novel approach for medical VQA that addresses this limitation by developing a model that can answer questions about image regions while considering the context necessary to answer the questions. Our experimental results demonstrate the effectiveness of our proposed model, outperforming existing methods on three datasets. Our code and data are available at https://github.com/sergiotasconmorales/locvqa.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Medical Visual Question Answering on RIS-VQA (test)](<https://www.sota2.com/research/sota/medical-visual-question-answering-on-ris-vqa-test.md>): Medical Visual Question Answering; RIS-VQA (test); AUC: 88.5.

- [Medical Visual Question Answering on INSEGCAT-VQA (test)](<https://www.sota2.com/research/sota/medical-visual-question-answering-on-insegcat-vqa-test.md>): Medical Visual Question Answering; INSEGCAT-VQA (test); AUC: 0.914.

- [Visual Question Answering on DME-VQA](<https://www.sota2.com/research/sota/visual-question-answering-on-dme-vqa.md>): Visual Question Answering; DME-VQA; Overall Accuracy: 84.2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2307.01067>)

[PDF](<https://arxiv.org/pdf/2307.01067.pdf>)