# Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding

Source: [Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding](<https://www.sota2.com/research/paper/68fa73131e209131cea5a54b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:57:06.308Z

Authors: Zheng Zhao, Emilio Monti, Jens Lehmann, Haytham Assem

Published: 2024-05-04T20:38:41.000Z

## Abstract

Large language models (LLMs) tend to inadequately integrate input context during text generation, relying excessively on encoded prior knowledge in model parameters, potentially resulting in generated text with factual inconsistencies or contextually unfaithful content. LLMs utilize two primary knowledge sources: 1) prior (parametric) knowledge from pretraining, and 2) contextual (non-parametric) knowledge from input prompts. The study addresses the open question of how LLMs effectively balance these knowledge sources during the generation process, specifically in the context of open-domain question answering. To address this issue, we introduce a novel approach integrating contrastive decoding with adversarial irrelevant passages as negative samples to enhance robust context grounding during generation. Notably, our method operates at inference time without requiring further training. We conduct comprehensive experiments to demonstrate its applicability and effectiveness, providing empirical evidence showcasing its superiority over existing methodologies. Our code is publicly available at: https://github.com/amazon-science/ContextualUnderstanding-ContrastiveDecoding.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Inter-Context Knowledge Conflict on StrategyQA](<https://www.sota2.com/research/sota/inter-context-knowledge-conflict-on-strategyqa.md>): Inter-Context Knowledge Conflict; StrategyQA; ACCL: 50.4.

- [Inter-Context Knowledge Conflict on PopQA](<https://www.sota2.com/research/sota/inter-context-knowledge-conflict-on-popqa.md>): Inter-Context Knowledge Conflict; PopQA; ACCL: 34.31.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2405.02750>)

[PDF](<https://arxiv.org/pdf/2405.02750.pdf>)