# BiRdQA: A Bilingual Dataset for Question Answering on Tricky Riddles

Source: [BiRdQA: A Bilingual Dataset for Question Answering on Tricky Riddles](<https://www.sota2.com/research/paper/68fa6c111e209131cea31e35>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:02:49.336Z

Authors: Yunxiang Zhang, Xiaojun Wan

Published: 2021-09-23T00:46:47.000Z

## Abstract

A riddle is a question or statement with double or veiled meanings, followed by an unexpected answer. Solving riddle is a challenging task for both machine and human, testing the capability of understanding figurative, creative natural language and reasoning with commonsense knowledge. We introduce BiRdQA, a bilingual multiple-choice question answering dataset with 6614 English riddles and 8751 Chinese riddles. For each riddle-answer pair, we provide four distractors with additional information from Wikipedia. The distractors are automatically generated at scale with minimal bias. Existing monolingual and multilingual QA models fail to perform well on our dataset, indicating that there is a long way to go before machine can beat human on solving tricky riddles. The dataset has been released to the community.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2109.11087>)

[PDF](<https://arxiv.org/pdf/2109.11087.pdf>)