# A Statistical Framework for Data-dependent Retrieval-Augmented Models

Source: [A Statistical Framework for Data-dependent Retrieval-Augmented Models](<https://www.sota2.com/research/paper/68fa74a91e209131cea6304d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:15:34.723Z

Authors: Soumya Basu, Ankit Singh Rawat, Manzil Zaheer

Published: 2024-08-27T20:51:06.000Z

## Abstract

Modern ML systems increasingly augment input instances with additional relevant information to enhance final prediction. Despite growing interest in such retrieval-augmented models, their fundamental properties and training are not well understood. We propose a statistical framework to study such models with two components: 1) a {\\em retriever} to identify the relevant information out of a large corpus via a data-dependent metric; and 2) a {\\em predictor} that consumes the input instances along with the retrieved information to make the final predictions. We present a principled method for end-to-end training of both components and draw connections with various training approaches in the literature. Furthermore, we establish excess risk bounds for retrieval-augmented models while delineating the contributions of both retriever and predictor towards the model performance. We validate the utility of our proposed training methods along with the key takeaways from our statistical analysis on open domain question answering task where retrieval augmentation is important.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2408.15399>)

[PDF](<https://arxiv.org/pdf/2408.15399.pdf>)
