# A Common Evaluation Setting for Just.Ask, Open Ephyra and Aranea QA systems

Source: [A Common Evaluation Setting for Just.Ask, Open Ephyra and Aranea QA systems](<https://www.sota2.com/research/paper/68fa608d1e209131ce9f016f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T19:42:41.418Z

Authors: Ricardo Pires

Published: 2012-05-08T19:19:32.000Z

## Abstract

Question Answering (QA) is not a new research field in Natural Language Processing (NLP). However in recent years, QA has been a subject of growing study. Nowadays, most of the QA systems have a similar pipelined architecture and each system use a set of unique techniques to accomplish its state of the art results. However, many things are not clear in the QA processing. It is not clear the extend of the impact of tasks performed in earlier stages in following stages of the pipelining process. It is not clear, if techniques used in a QA system can be used in another QA system to improve its results. And finally, it is not clear in what setting should be these systems tested in order to properly analyze their results.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1205.1779>)

[PDF](<https://arxiv.org/pdf/1205.1779.pdf>)
