# CARAFE: Content-Aware ReAssembly of FEatures

Source: [CARAFE: Content-Aware ReAssembly of FEatures](<https://www.sota2.com/research/paper/68fa66911e209131cea10f20>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:09:32.565Z

Authors: Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, Dahua Lin

Published: 2019-05-06T17:58:06.000Z

## Abstract

Feature upsampling is a key operation in a number of modern convolutional network architectures, e.g. feature pyramids. Its design is critical for dense prediction tasks such as object detection and semantic/instance segmentation. In this work, we propose Content-Aware ReAssembly of FEatures (CARAFE), a universal, lightweight and highly effective operator to fulfill this goal. CARAFE has several appealing properties: (1) Large field of view. Unlike previous works (e.g. bilinear interpolation) that only exploit sub-pixel neighborhood, CARAFE can aggregate contextual information within a large receptive field. (2) Content-aware handling. Instead of using a fixed kernel for all samples (e.g. deconvolution), CARAFE enables instance-specific content-aware handling, which generates adaptive kernels on-the-fly. (3) Lightweight and fast to compute. CARAFE introduces little computational overhead and can be readily integrated into modern network architectures. We conduct comprehensive evaluations on standard benchmarks in object detection, instance/semantic segmentation and inpainting. CARAFE shows consistent and substantial gains across all the tasks (1.2%, 1.3%, 1.8%, 1.1db respectively) with negligible computational overhead. It has great potential to serve as a strong building block for future research. It has great potential to serve as a strong building block for future research. Code and models are available at https://github.com/open-mmlab/mmdetection.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Inpainting on Places (val)](<https://www.sota2.com/research/sota/image-inpainting-on-places-val.md>): Image Inpainting; Places (val); L1 Distance: 0.0572.

- [Object Detection on MS COCO 2018 (test-dev)](<https://www.sota2.com/research/sota/object-detection-on-ms-coco-2018-test-dev.md>): Object Detection; MS-COCO 2018 (test-dev); AP: 38.8.

- [Instance Segmentation on MS COCO 2018 (test-dev)](<https://www.sota2.com/research/sota/instance-segmentation-on-ms-coco-2018-test-dev.md>): Instance Segmentation; MS-COCO 2018 (test-dev); AP: 35.9.

- [Grad-CAM Evaluation on ImageNet](<https://www.sota2.com/research/sota/grad-cam-evaluation-on-imagenet.md>): Grad-CAM Evaluation; ImageNet; A.D: 49.9.

- [CAM Faithfulness on ImageNet (val)](<https://www.sota2.com/research/sota/cam-faithfulness-on-imagenet-val.md>): CAM Faithfulness; ImageNet (val); Average Drop: 10.24.

- [Depth Estimation on COCO pseudo-labels from Depth Anything V2 (val)](<https://www.sota2.com/research/sota/depth-estimation-on-coco-pseudo-labels-from-depth-anything-v2-val.md>): Depth Estimation; COCO pseudo-labels from Depth Anything V2 (val); Delta 1: 61.42.

- [Medical Image Segmentation on ACDC (Structure Dice and HD95)](<https://www.sota2.com/research/sota/medical-image-segmentation-on-acdc-structure-dice-and-hd95.md>): Medical Image Segmentation; ACDC; RV Dice: 87.57.

- [Semantic Segmentation on Cityscapes 27 classes (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-cityscapes-27-classes-val.md>): Semantic segmentation; Cityscapes 27 classes (val); mIoU: 56.05.

- [Semantic Segmentation on ADE20K 150 classes (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k-150-classes-val.md>): Semantic segmentation; ADE20K 150 classes (val); mIoU: 38.3.

- [Semantic Segmentation on COCO Stuff-27 (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-coco-stuff-27-val.md>): Semantic segmentation; COCO Stuff-27 (val); mIoU: 59.73.

- [Depth Estimation on NYU v2 (val)](<https://www.sota2.com/research/sota/depth-estimation-on-nyu-v2-val.md>): Depth Estimation; NYU v2 (val); RMSE: 1.09.

- [Semantic Segmentation on ADE20K](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k.md>): Semantic segmentation; ADE20K; mIoU: 51.85.

- [Semantic Segmentation on Pascal VOC 21 classes (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-21-classes-val.md>): Semantic segmentation; Pascal VOC 21 classes (val); mIoU: 0.8026.

- [Semantic Segmentation on ADE20K (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k-val.md>): Semantic segmentation; ADE20K (val); mIoU: 54.48.

- [Panoptic Segmentation on COCO 2017 (val)](<https://www.sota2.com/research/sota/panoptic-segmentation-on-coco-2017-val.md>): Panoptic Segmentation; COCO 2017 (val); PQ: 40.8.

- [Semantic Segmentation on PASCAL VOC (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-val.md>): Semantic segmentation; PASCAL VOC (val); mIoU: 42.39.

- [Instance Segmentation on COCO 2017 (val)](<https://www.sota2.com/research/sota/instance-segmentation-on-coco-2017-val.md>): Instance Segmentation; COCO 2017 (val); APm: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1905.02188>)

[PDF](<https://arxiv.org/pdf/1905.02188.pdf>)

[Implementation](<https://github.com/open-mmlab/mmdetection>)
