# RISAM: Referring Image Segmentation via Mutual-Aware Attention Features Source: [RISAM: Referring Image Segmentation via Mutual-Aware Attention Features]() Collection index: [llms.txt]() Data updated: 2025-10-27T06:34:03.698Z Authors: Mengxi Zhang, Yiming Liu, Xiangjun Yin, Huanjing Yue, Jingyu Yang Published: 2023-11-27T11:24:25.000Z ## Abstract Referring image segmentation (RIS) aims to segment a particular region based on a language expression prompt. Existing methods incorporate linguistic features into visual features and obtain multi-modal features for mask decoding. However, these methods may segment the visually salient entity instead of the correct referring region, as the multi-modal features are dominated by the abundant visual context. In this paper, we propose MARIS, a referring image segmentation method that leverages the Segment Anything Model (SAM) and introduces a mutual-aware attention mechanism to enhance the cross-modal fusion via two parallel branches. Specifically, our mutual-aware attention mechanism consists of Vision-Guided Attention and Language-Guided Attention, which bidirectionally model the relationship between visual and linguistic features. Correspondingly, we design a Mask Decoder to enable explicit linguistic guidance for more consistent segmentation with the language expression. To this end, a multi-modal query token is proposed to integrate linguistic information and interact with visual information simultaneously. Extensive experiments on three benchmark datasets show that our method outperforms the state-of-the-art RIS methods. Our code will be publicly available. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Generalized Referring Image Segmentation on PhraseCut (test)](): Generalized Referring Image Segmentation; PhraseCut (test); gIoU: 22.82. - [Referring Image Segmentation on RefCOCO (val)](): Referring Image Segmentation; RefCOCO (val); mIoU: not recorded. - [Referring Image Segmentation on RefCOCO+ (test B)](): Referring Image Segmentation; RefCOCO+ (test-B); mIoU: not recorded. - [Referring Image Segmentation on RefCOCO (test A)](): Referring Image Segmentation; RefCOCO (test A); mIoU: not recorded. - [Referring Image Segmentation on RefCOCO+ (val)](): Referring Image Segmentation; RefCOCO+ (val); mIoU: not recorded. - [Referring Image Segmentation on RefCOCO (test B)](): Referring Image Segmentation; RefCOCO (test-B); mIoU: not recorded. - [Referring Image Segmentation on RefCOCO+ (testA)](): Referring Image Segmentation; RefCOCO+ (testA); mIoU: not recorded. - [Referring Image Segmentation on RefCOCOg (val)](): Referring Image Segmentation; RefCOCOg (val); oIoU: not recorded. - [Referring Image Segmentation on RefCOCOg (test)](): Referring Image Segmentation; RefCOCOg (test); oIoU: not recorded. ## Original sources and implementation [Original paper]() [PDF]()