# ResAgent: Entropy-based Prior Point Discovery and Visual Reasoning for Referring Expression Segmentation

Source: [ResAgent: Entropy-based Prior Point Discovery and Visual Reasoning for Referring Expression Segmentation](<https://www.sota2.com/research/paper/6976f526a2b28e761d87c2cf>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-27T05:00:04.746Z

Authors: Yihao Wang, Jusheng Zhang, Ziyi Tang, Keze Wang, Meng Yang

Published: 2026-01-23T01:56:04.000Z

## Abstract

Referring Expression Segmentation (RES) is a core vision-language segmentation task that enables pixel-level understanding of targets via free-form linguistic expressions, supporting critical applications such as human-robot interaction and augmented reality. Despite the progress of Multimodal Large Language Model (MLLM)-based approaches, existing RES methods still suffer from two key limitations: first, the coarse bounding boxes from MLLMs lead to redundant or non-discriminative point prompts; second, the prevalent reliance on textual coordinate reasoning is unreliable, as it fails to distinguish targets from visually similar distractors. To address these issues, we propose \\textbf{\\model}, a novel RES framework integrating \\textbf{E}ntropy-\\textbf{B}ased Point \\textbf{D}iscovery (\\textbf{EBD}) and \\textbf{V}ision-\\textbf{B}ased \\textbf{R}easoning (\\textbf{VBR}). Specifically, EBD identifies high-information candidate points by modeling spatial uncertainty within coarse bounding boxes, treating point selection as an information maximization process. VBR verifies point correctness through joint visual-semantic alignment, abandoning text-only coordinate inference for more robust validation. Built on these components, \\model implements a coarse-to-fine workflow: bounding box initialization, entropy-guided point discovery, vision-based validation, and mask decoding. Extensive evaluations on four benchmark datasets (RefCOCO, RefCOCO+, RefCOCOg, and ReasonSeg) demonstrate that \\model achieves new state-of-the-art performance across all four benchmarks, highlighting its effectiveness in generating accurate and semantically grounded segmentation masks with minimal prompts.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Reasoning Segmentation on ReasonSeg (val)](<https://www.sota2.com/research/sota/reasoning-segmentation-on-reasonseg-val.md>): Reasoning Segmentation; ReasonSeg (val); gIoU: 72.74.

- [Referring Expression Segmentation on RefCOCO (testA)](<https://www.sota2.com/research/sota/referring-expression-segmentation-on-refcoco-testa.md>): Referring Expression Segmentation; RefCOCO (testA); cIoU: not recorded.

- [Referring Expression Segmentation on RefCOCO+ (testA)](<https://www.sota2.com/research/sota/referring-expression-segmentation-on-refcoco-plus-testa.md>): Referring Expression Segmentation; RefCOCO+ (testA); cIoU: not recorded.

- [Referring Expression Segmentation on RefCOCO+ (val)](<https://www.sota2.com/research/sota/referring-expression-segmentation-on-refcoco-plus-val.md>): Referring Expression Segmentation; RefCOCO+ (val); cIoU: not recorded.

- [Referring Expression Segmentation on RefCOCO (val)](<https://www.sota2.com/research/sota/referring-expression-segmentation-on-refcoco-val.md>): Referring Expression Segmentation; RefCOCO (val); cIoU: not recorded.

- [Referring Expression Segmentation on RefCOCO (testB)](<https://www.sota2.com/research/sota/referring-expression-segmentation-on-refcoco-testb.md>): Referring Expression Segmentation; RefCOCO (testB); cIoU: not recorded.

- [Referring Expression Segmentation on RefCOCO+ (testB)](<https://www.sota2.com/research/sota/referring-expression-segmentation-on-refcoco-plus-testb.md>): Referring Expression Segmentation; RefCOCO+ (testB); cIoU: not recorded.

- [Referring Expression Segmentation on RefCOCOg (val-u)](<https://www.sota2.com/research/sota/referring-expression-segmentation-on-refcocog-val-u.md>): Referring Expression Segmentation; RefCOCOg (val (U)); cIoU: not recorded.

- [Referring Expression Segmentation on RefCOCOg (test-u)](<https://www.sota2.com/research/sota/referring-expression-segmentation-on-refcocog-test-u.md>): Referring Expression Segmentation; RefCOCOg (test(U)); cIoU: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.16394>)

[PDF](<https://arxiv.org/pdf/2601.16394.pdf>)
