# Decoupled Context Processing for Context Augmented Language Modeling

Source: [Decoupled Context Processing for Context Augmented Language Modeling](<https://www.sota2.com/research/paper/68fa6d9e1e209131cea3adb6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:38:22.399Z

Authors: Zonglin Li, Ruiqi Guo, Sanjiv Kumar

Published: 2022-10-11T20:05:09.000Z

## Abstract

Language models can be augmented with a context retriever to incorporate knowledge from large external databases. By leveraging retrieved context, the neural network does not have to memorize the massive amount of world knowledge within its internal parameters, leading to better parameter efficiency, interpretability and modularity. In this paper we examined a simple yet effective architecture for incorporating external context into language models based on decoupled Encoder Decoder architecture. We showed that such a simple architecture achieves competitive results on auto-regressive language modeling and open domain question answering tasks. We also analyzed the behavior of the proposed model which performs grounded context transfer. Finally we discussed the computational implications of such retrieval augmented models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2210.05758>)

[PDF](<https://arxiv.org/pdf/2210.05758.pdf>)
