# MDSAM:Memory-Driven Sparse Attention Matrix for LVLMs Hallucination Mitigation

Source: [MDSAM:Memory-Driven Sparse Attention Matrix for LVLMs Hallucination Mitigation](<https://www.sota2.com/research/paper/68fa798d1e209131cea7e2b6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:00:46.441Z

Authors: Shuaiye Lu, Linjiang Zhou, Xiaochuan Shi

Published: 2025-06-21T09:49:16.000Z

## Abstract

Hallucinations in large vision-language models (LVLMs) often stem from the model's sensitivity to image tokens during decoding, as evidenced by attention peaks observed when generating both real and hallucinated entities. To address this, we propose Memory-Driven Sparse Attention Matrix (MDSAM) , a novel training-free approach that dynamically captures and refines the attention allocated to image tokens at each layer. MDSAM memorizes attention patterns and activates updates through alignment during decoding, enhancing focus on relevant image tokens while effectively reducing hallucinations. We evaluate MDSAM on multiple benchmarks for tasks such as image captioning and visual question answering, demonstrating its ability to consistently reduce hallucinations and improve reliability. Compatible with various LVLM architectures, MDSAM highlights its adaptability and effectiveness in mitigating hallucinations without requiring additional training or external tools.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2506.17664>)

[PDF](<https://arxiv.org/pdf/2506.17664.pdf>)
