# Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering

Source: [Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering](<https://www.sota2.com/research/paper/68fa6d921e209131cea3ab90>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:38:22.399Z

Authors: Priyanka Sen, Alham Fikri Aji, Amir Saffari

Published: 2022-10-04T13:54:29.000Z

## Abstract

We introduce Mintaka, a complex, natural, and multilingual dataset designed for experimenting with end-to-end question-answering models. Mintaka is composed of 20,000 question-answer pairs collected in English, annotated with Wikidata entities, and translated into Arabic, French, German, Hindi, Italian, Japanese, Portuguese, and Spanish for a total of 180,000 samples. Mintaka includes 8 types of complex questions, including superlative, intersection, and multi-hop questions, which were naturally elicited from crowd workers. We run baselines over Mintaka, the best of which achieves 38% hits@1 in English and 31% hits@1 multilingually, showing that existing models have room for improvement. We release Mintaka at https://github.com/amazon-research/mintaka.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2210.01613>)

[PDF](<https://arxiv.org/pdf/2210.01613.pdf>)