# ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer

Source: [ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer](<https://www.sota2.com/research/paper/68fa792e1e209131cea7c76e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-14T19:52:34.832Z

Authors: Youwei Pang, Xiaoqi Zhao, Lihe Zhang, Huchuan Lu

Published: 2023-07-23T15:17:45.000Z

## Abstract

Deep learning (DL) has advanced the field of dense prediction, while gradually dissolving the inherent barriers between different tasks. However, most existing works focus on designing architectures and constructing visual cues only for the specific task, which ignores the potential uniformity introduced by the DL paradigm. In this paper, we attempt to construct a novel $\\underline{ComP}$lementary $\\underline{tr}$ansformer, $\\textbf{ComPtr}$, for diverse bi-source dense prediction tasks. Specifically, unlike existing methods that over-specialize in a single task or a subset of tasks, ComPtr starts from the more general concept of bi-source dense prediction. Based on the basic dependence on information complementarity, we propose consistency enhancement and difference awareness components with which ComPtr can evacuate and collect important visual semantic cues from different image sources for diverse tasks, respectively. ComPtr treats different inputs equally and builds an efficient dense interaction model in the form of sequence-to-sequence on top of the transformer. This task-generic design provides a smooth foundation for constructing the unified model that can simultaneously deal with various bi-source information. In extensive experiments across several representative vision tasks, i.e. remote sensing change detection, RGB-T crowd counting, RGB-D/T salient object detection, and RGB-D semantic segmentation, the proposed method consistently obtains favorable performance. The code will be available at https://github.com/lartpang/ComPtr.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Semantic Segmentation on NYU v2 (test)](<https://www.sota2.com/research/sota/semantic-segmentation-on-nyu-v2-test.md>): Semantic segmentation; NYU v2 (test); mIoU: 49.2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2307.12349>)

[PDF](<https://arxiv.org/pdf/2307.12349.pdf>)

[Implementation](<https://github.com/lartpang/ComPtr>)
