# MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models

Source: [MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models](<https://www.sota2.com/research/paper/68fa67741e209131cea16623>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:15:13.389Z

Authors: Mandy Guo, Yinfei Yang, Daniel Cer, Qinlan Shen, Noah Constant

Published: 2020-05-05T21:30:16.000Z

## Abstract

Retrieval question answering (ReQA) is the task of retrieving a sentence-level answer to a question from an open corpus (Ahmad et al.,2019).This paper presents MultiReQA, anew multi-domain ReQA evaluation suite com-posed of eight retrieval QA tasks drawn from publicly available QA datasets. We provide the first systematic retrieval based evaluation over these datasets using two supervised neural models, based on fine-tuning BERT andUSE-QA models respectively, as well as a surprisingly strong information retrieval baseline,BM25. Five of these tasks contain both train-ing and test data, while three contain test data only. Performance on the five tasks with train-ing data shows that while a general model covering all domains is achievable, the best performance is often obtained by training exclusively on in-domain data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2005.02507>)

[PDF](<https://arxiv.org/pdf/2005.02507.pdf>)