# TRAQ: Trustworthy Retrieval Augmented Question Answering via Conformal Prediction Source: [TRAQ: Trustworthy Retrieval Augmented Question Answering via Conformal Prediction]() Collection index: [llms.txt]() Data updated: 2025-10-26T02:35:49.064Z Authors: Shuo Li, Sangdon Park, Insup Lee, Osbert Bastani Published: 2023-07-07T02:42:06.000Z ## Abstract When applied to open-domain question answering, large language models (LLMs) frequently generate incorrect responses based on made-up facts, which are called $\\textit{hallucinations}$. Retrieval augmented generation (RAG) is a promising strategy to avoid hallucinations, but it does not provide guarantees on its correctness. To address this challenge, we propose the Trustworthy Retrieval Augmented Question Answering, or $\\textit{TRAQ}$, which provides the first end-to-end statistical correctness guarantee for RAG. TRAQ uses conformal prediction, a statistical technique for constructing prediction sets that are guaranteed to contain the semantically correct response with high probability. Additionally, TRAQ leverages Bayesian optimization to minimize the size of the constructed sets. In an extensive experimental evaluation, we demonstrate that TRAQ provides the desired correctness guarantee while reducing prediction set size by 16.2% on average compared to an ablation. The implementation is available at $\\href{https://github.com/shuoli90/TRAQ.git}{TRAQ}$. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()