# RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder

Source: [RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder](<https://www.sota2.com/research/paper/68fa689f1e209131cea1d2d6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:23:13.681Z

Authors: Cheng Chi, Fangyun Wei, Han Hu

Published: 2020-10-29T17:59:45.000Z

## Abstract

Existing object detection frameworks are usually built on a single format of object/part representation, i.e., anchor/proposal rectangle boxes in RetinaNet and Faster R-CNN, center points in FCOS and RepPoints, and corner points in CornerNet. While these different representations usually drive the frameworks to perform well in different aspects, e.g., better classification or finer localization, it is in general difficult to combine these representations in a single framework to make good use of each strength, due to the heterogeneous or non-grid feature extraction by different representations. This paper presents an attention-based decoder module similar as that in Transformer~\\cite{vaswani2017attention} to bridge other representations into a typical object detector built on a single representation format, in an end-to-end fashion. The other representations act as a set of \\emph{key} instances to strengthen the main \\emph{query} representation features in the vanilla detectors. Novel techniques are proposed towards efficient computation of the decoder module, including a \\emph{key sampling} approach and a \\emph{shared location embedding} approach. The proposed module is named \\emph{bridging visual representations} (BVR). It can perform in-place and we demonstrate its broad effectiveness in bridging other representations into prevalent object detection frameworks, including RetinaNet, Faster R-CNN, FCOS and ATSS, where about $1.5\\sim3.0$ AP improvements are achieved. In particular, we improve a state-of-the-art framework with a strong backbone by about $2.0$ AP, reaching $52.7$ AP on COCO test-dev. The resulting network is named RelationNet++. The code will be available at https://github.com/microsoft/RelationNet2.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Detection on MS COCO (test-dev)](<https://www.sota2.com/research/sota/object-detection-on-ms-coco-test-dev.md>): Object Detection; MS COCO (test-dev); mAP@.5: 70.4.

- [Object Detection on COCO (test-dev)](<https://www.sota2.com/research/sota/object-detection-on-coco-test-dev.md>): Object Detection; COCO (test-dev); mAP: 52.7.

- [Object Detection on COCO 2017 (val)](<https://www.sota2.com/research/sota/object-detection-on-coco-2017-val.md>): Object Detection; COCO 2017 (val); AP: 39.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2010.15831>)

[PDF](<https://arxiv.org/pdf/2010.15831.pdf>)

[Implementation](<https://github.com/microsoft/RelationNet2>)