# Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering

Source: [Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering](<https://www.sota2.com/research/paper/68fa65031e209131cea07deb>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:16:52.800Z

Authors: Medhini Narasimhan, Svetlana Lazebnik, Alexander G. Schwing

Published: 2018-11-01T17:59:56.000Z

## Abstract

Accurately answering a question about a given image requires combining observations with general knowledge. While this is effortless for humans, reasoning with general knowledge remains an algorithmic challenge. To advance research in this direction a novel \`fact-based' visual question answering (FVQA) task has been introduced recently along with a large set of curated facts which link two entities, i.e., two possible answers, via a relation. Given a question-image pair, deep network techniques have been employed to successively reduce the large set of facts until one of the two entities of the final remaining fact is predicted as the answer. We observe that a successive process which considers one fact at a time to form a local decision is sub-optimal. Instead, we develop an entity graph and use a graph convolutional network to \`reason' about the correct answer by jointly considering all entities. We show on the challenging FVQA dataset that this leads to an improvement in accuracy of around 7% compared to the state of the art.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1811.00538>)

[PDF](<https://arxiv.org/pdf/1811.00538.pdf>)