# WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentation Source: [WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentation]() Collection index: [llms.txt]() Data updated: 2025-12-22T11:40:39.236Z Authors: Yang Liu, Silin Cheng, Xinwei He, Sebastien Ourselin, Lei Tan, Gen Luo Published: 2025-05-24T13:05:17.000Z ## Abstract Weakly supervised referring expression comprehension(WREC) and segmentation(WRES) aim to learn object grounding based on a given expression using weak supervision signals like image-text pairs. While these tasks have traditionally been modeled separately, we argue that they can benefit from joint learning in a multi-task framework. To this end, we propose WeakMCN, a novel multi-task collaborative network that effectively combines WREC and WRES with a dual-branch architecture. Specifically, the WREC branch is formulated as anchor-based contrastive learning, which also acts as a teacher to supervise the WRES branch. In WeakMCN, we propose two innovative designs to facilitate multi-task collaboration, namely Dynamic Visual Feature Enhancement(DVFE) and Collaborative Consistency Module(CCM). DVFE dynamically combines various pre-trained visual knowledge to meet different task requirements, while CCM promotes cross-task consistency from the perspective of optimization. Extensive experimental results on three popular REC and RES benchmarks, i.e., RefCOCO, RefCOCO+, and RefCOCOg, consistently demonstrate performance gains of WeakMCN over state-of-the-art single-task alternatives, e.g., up to 3.91% and 13.11% on RefCOCO for WREC and WRES tasks, respectively. Furthermore, experiments also validate the strong generalization ability of WeakMCN in both semi-supervised REC and RES settings against existing methods, e.g., +8.94% for semi-REC and +7.71% for semi-RES on 1% RefCOCO. The code is publicly available at https://github.com/MRUIL/WeakMCN. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Weakly Supervised Referring Expression Comprehension on RefCOCO (val)](): Weakly Supervised Referring Expression Comprehension; RefCOCO (val); Accuracy: 69.2. - [Weakly Supervised Referring Expression Comprehension on RefCOCO (testB)](): Weakly Supervised Referring Expression Comprehension; RefCOCO (testB); Accuracy: 62.63. - [Weakly Supervised Referring Expression Comprehension on RefCOCO+ (val)](): Weakly Supervised Referring Expression Comprehension; RefCOCO+ (val); Accuracy: 51.9. - [Weakly Supervised Referring Expression Comprehension on RefCOCO+ (testA)](): Weakly Supervised Referring Expression Comprehension; RefCOCO+ (testA); Accuracy: 57.33. - [Weakly Supervised Referring Expression Comprehension on RefCOCO+ (testB)](): Weakly Supervised Referring Expression Comprehension; RefCOCO+ (testB); Accuracy: 43.1. - [Weakly Supervised Referring Expression Comprehension on RefCOCOg (val)](): Weakly Supervised Referring Expression Comprehension; RefCOCOg (val); Accuracy: 54.62. - [Referring Expression Comprehension on RefCOCO (testA)](): Referring Expression Comprehension; RefCOCO (testA); Accuracy: 0.7078. - [Referring Expression Comprehension on RefCOCO+ (val)](): Referring Expression Comprehension; RefCOCO+ (val); Accuracy: not recorded. - [Referring Expression Comprehension on RefCOCO (val)](): Referring Expression Comprehension; RefCOCO (val); Accuracy: not recorded. - [Referring Expression Comprehension on RefCOCOg (val)](): Referring Expression Comprehension; RefCOCOg (val); Accuracy: not recorded. - [Referring Expression Comprehension on RefCOCO+ (testB)](): Referring Expression Comprehension; RefCOCO+ (testB); Accuracy: not recorded. - [Referring Expression Comprehension on RefCOCO+ (testA)](): Referring Expression Comprehension; RefCOCO+ (testA); Accuracy: not recorded. - [Referring Expression Comprehension on RefCOCO (testB)](): Referring Expression Comprehension; RefCOCO (testB); Accuracy: not recorded. ## Original sources and implementation [Original paper]() [PDF]()