# Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?

Source: [Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?](<https://www.sota2.com/research/paper/68fa62841e209131ce9f9cc4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:21:33.904Z

Authors: Abhishek Das, Harsh Agrawal, C. Lawrence Zitnick, Devi Parikh, Dhruv Batra

Published: 2016-06-17T17:00:02.000Z

## Abstract

We conduct large-scale studies on \`human attention' in Visual Question Answering (VQA) to understand where humans choose to look to answer questions about images. We design and test multiple game-inspired novel attention-annotation interfaces that require the subject to sharpen regions of a blurred image to answer a question. Thus, we introduce the VQA-HAT (Human ATtention) dataset. We evaluate attention maps generated by state-of-the-art VQA models against human attention both qualitatively (via visualizations) and quantitatively (via rank-order correlation). Overall, our experiments show that current attention models in VQA do not seem to be looking at the same regions as humans.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1606.05589>)

[PDF](<https://arxiv.org/pdf/1606.05589.pdf>)