# iSeg: An Iterative Refinement-based Framework for Training-free Segmentation

Source: [iSeg: An Iterative Refinement-based Framework for Training-free Segmentation](<https://www.sota2.com/research/paper/68fa75211e209131cea65fee>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:27:16.108Z

Authors: Lin Sun, Jiale Cao, Jin Xie, Fahad Shahbaz Khan, Yanwei Pang

Published: 2024-09-05T03:07:26.000Z

## Abstract

Stable diffusion has demonstrated strong image synthesis ability to given text descriptions, suggesting it to contain strong semantic clue for grouping objects. The researchers have explored employing stable diffusion for training-free segmentation. Most existing approaches refine cross-attention map by self-attention map once, demonstrating that self-attention map contains useful semantic information to improve segmentation. To fully utilize self-attention map, we present a deep experimental analysis on iteratively refining cross-attention map with self-attention map, and propose an effective iterative refinement framework for training-free segmentation, named iSeg. The proposed iSeg introduces an entropy-reduced self-attention module that utilizes a gradient descent scheme to reduce the entropy of self-attention map, thereby suppressing the weak responses corresponding to irrelevant global information. Leveraging the entropy-reduced self-attention module, our iSeg stably improves refined cross-attention map with iterative refinement. Further, we design a category-enhanced cross-attention module to generate accurate cross-attention map, providing a better initial input for iterative refinement. Extensive experiments across different datasets and diverse segmentation tasks reveal the merits of proposed contributions, leading to promising performance on diverse segmentation tasks. For unsupervised semantic segmentation on Cityscapes, our iSeg achieves an absolute gain of 3.8% in terms of mIoU compared to the best existing training-free approach in literature. Moreover, our proposed iSeg can support segmentation with different kinds of images and interactions. The project is available at https://linsun449.github.io/iSeg.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Semantic Segmentation on COCO-Stuff 27](<https://www.sota2.com/research/sota/semantic-segmentation-on-coco-stuff-27.md>): Semantic segmentation; COCO-Stuff 27; mIoU: 45.2.

- [Semantic Segmentation on Cityscapes](<https://www.sota2.com/research/sota/semantic-segmentation-on-cityscapes.md>): Semantic segmentation; Cityscapes; mIoU: 25.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2409.03209>)

[PDF](<https://arxiv.org/pdf/2409.03209.pdf>)
