# CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset Source: [CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset]() Collection index: [llms.txt]() Data updated: 2026-02-04T05:00:24.095Z Authors: Jind\\v{r}ich Libovick\\'y, Jind\\v{r}ich Helcl, Andrei Manea, Gianluca Vico Published: 2025-07-30T15:10:55.000Z ## Abstract We introduce CUS-QA, a benchmark for evaluation of open-ended regional question answering that encompasses both textual and visual modalities. We also provide strong baselines using state-of-the-art large language models (LLMs). Our dataset consists of manually curated questions and answers grounded in Wikipedia, created by native speakers from Czechia, Slovakia, and Ukraine, with accompanying English translations. It includes both purely textual questions and those requiring visual understanding. We evaluate state-of-the-art LLMs through prompting and add human judgments of answer correctness. Using these human evaluations, we analyze the reliability of existing automatic evaluation metrics. Our baseline results show that even the best open-weight LLMs achieve only over 40% accuracy on textual questions and below 30% on visual questions. LLM-based evaluation metrics show strong correlation with human judgment, while traditional string-overlap metrics perform surprisingly well due to the prevalence of named entities in answers. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()