# FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation Source: [FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation]() Collection index: [llms.txt]() Data updated: 2025-10-26T03:55:27.100Z Authors: Bingyu Li, Da Zhang, Zhiyuan Zhao, Junyu Gao, Xuelong Li Published: 2025-01-01T15:47:04.000Z ## Abstract Open-vocabulary segmentation aims to identify and segment specific regions and objects based on text-based descriptions. A common solution is to leverage powerful vision-language models (VLMs), such as CLIP, to bridge the gap between vision and text information. However, VLMs are typically pretrained for image-level vision-text alignment, focusing on global semantic features. In contrast, segmentation tasks require fine-grained pixel-level alignment and detailed category boundary information, which VLMs alone cannot provide. As a result, information extracted directly from VLMs can't meet the requirements of segmentation tasks. To address this limitation, we propose FGAseg, a model designed for fine-grained pixel-text alignment and category boundary supplementation. The core of FGAseg is a Pixel-Level Alignment module that employs a cross-modal attention mechanism and a text-pixel alignment loss to refine the coarse-grained alignment from CLIP, achieving finer-grained pixel-text semantic alignment. Additionally, to enrich category boundary information, we introduce the alignment matrices as optimizable pseudo-masks during forward propagation and propose Category Information Supplementation module. These pseudo-masks, derived from cosine and convolutional similarity, provide essential global and local boundary information between different categories. By combining these two strategies, FGAseg effectively enhances pixel-level alignment and category boundary information, addressing key challenges in open-vocabulary segmentation. Extensive experiments demonstrate that FGAseg outperforms existing methods on open-vocabulary semantic segmentation benchmarks. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Open-Vocabulary Semantic Segmentation on DDHR-SK to DDHR-CH](): Open Vocabulary Semantic Segmentation; DDHR-SK to DDHR-CH; mIoU: 40.6. - [Open-Vocabulary Semantic Segmentation on OpenEarthMap-SAR OEM-thin](): Open Vocabulary Semantic Segmentation; OpenEarthMap-SAR OEM-thin; mIoU: 32.8. - [Open-Vocabulary Semantic Segmentation on PIE-RGB-SAR cloud](): Open Vocabulary Semantic Segmentation; PIE-RGB-SAR cloud; mIoU: 51.6. - [Open-Vocabulary Semantic Segmentation on DDHR-SK](): Open Vocabulary Semantic Segmentation; DDHR-SK; mIoU: 51.6. - [Open-Vocabulary Semantic Segmentation on PIE-RGB-SAR clean](): Open Vocabulary Semantic Segmentation; PIE-RGB-SAR clean; mIoU: 52.1. - [Open-Vocabulary Semantic Segmentation on MM-OVSeg Evaluation Suite Combined](): Open Vocabulary Semantic Segmentation; MM-OVSeg Evaluation Suite Combined; mIoU: 42.5. - [Open-Vocabulary Semantic Segmentation on OpenEarthMap-SAR thick](): Open Vocabulary Semantic Segmentation; OpenEarthMap-SAR thick; mIoU: 26. ## Original sources and implementation [Original paper]() [PDF]()