# CS563-QA: A Collection for Evaluating Question Answering Systems

Source: [CS563-QA: A Collection for Evaluating Question Answering Systems](<https://www.sota2.com/research/paper/68fa693c1e209131cea210d8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:56:51.523Z

Authors: Katerina Papantoniou, Yannis Tzitzikas

Published: 2019-07-02T20:00:29.000Z

## Abstract

Question Answering (QA) is a challenging topic since it requires tackling the various difficulties of natural language understanding. Since evaluation is important not only for identifying the strong and weak points of the various techniques for QA, but also for facilitating the inception of new methods and techniques, in this paper we present a collection for evaluating QA methods over free text that we have created. Although it is a small collection, it contains cases of increasing difficulty, therefore it has an educational value and it can be used for rapid evaluation of QA systems.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1907.01611>)

[PDF](<https://arxiv.org/pdf/1907.01611.pdf>)