# Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering

Source: [Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering](<https://www.sota2.com/research/paper/68fa6e221e209131cea3e0f2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:50:11.375Z

Authors: AJ Piergiovanni, Wei Li, Weicheng Kuo, Mohammad Saffar, Fred Bertsch, Anelia Angelova

Published: 2022-05-02T14:53:13.000Z

## Abstract

We present Answer-Me, a task-aware multi-task framework which unifies a variety of question answering tasks, such as, visual question answering, visual entailment, visual reasoning. In contrast to previous works using contrastive or generative captioning training, we propose a novel and simple recipe to pre-train a vision-language joint model, which is multi-task as well. The pre-training uses only noisy image captioning data, and is formulated to use the entire architecture end-to-end with both a strong language encoder and decoder. Our results show state-of-the-art performance, zero-shot generalization, robustness to forgetting, and competitive single-task results across a variety of question answering tasks. Our multi-task mixture training learns from tasks of various question intents and thus generalizes better, including on zero-shot vision-language tasks. We conduct experiments in the challenging multi-task and open-vocabulary settings and across a variety of datasets and tasks, such as VQA2.0, SNLI-VE, NLVR2, GQA. We observe that the proposed approach is able to generalize to unseen tasks and that more diverse mixtures lead to higher accuracy in both known and novel tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Visual Question Answering on VQA 2.0 (test-dev)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-2-0-test-dev.md>): Visual Question Answering; VQA 2.0 (test-dev); Accuracy: 73.6.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2205.00949>)

[PDF](<https://arxiv.org/pdf/2205.00949.pdf>)