# Towards Efficient Use of Multi-Scale Features in Transformer-Based Object Detectors

Source: [Towards Efficient Use of Multi-Scale Features in Transformer-Based Object Detectors](<https://www.sota2.com/research/paper/68fa6ee21e209131cea42a2b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:23:10.942Z

Authors: Gongjie Zhang, Zhipeng Luo, Zichen Tian, Jingyi Zhang, Xiaoqin Zhang, Shijian Lu

Published: 2022-08-24T08:09:25.000Z

## Abstract

Multi-scale features have been proven highly effective for object detection but often come with huge and even prohibitive extra computation costs, especially for the recent Transformer-based detectors. In this paper, we propose Iterative Multi-scale Feature Aggregation (IMFA) -- a generic paradigm that enables efficient use of multi-scale features in Transformer-based object detectors. The core idea is to exploit sparse multi-scale features from just a few crucial locations, and it is achieved with two novel designs. First, IMFA rearranges the Transformer encoder-decoder pipeline so that the encoded features can be iteratively updated based on the detection predictions. Second, IMFA sparsely samples scale-adaptive features for refined detection from just a few keypoint locations under the guidance of prior detection predictions. As a result, the sampled multi-scale features are sparse yet still highly beneficial for object detection. Extensive experiments show that the proposed IMFA boosts the performance of multiple Transformer-based object detectors significantly yet with only slight computational overhead.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Detection on COCO 2017 (val)](<https://www.sota2.com/research/sota/object-detection-on-coco-2017-val.md>): Object Detection; COCO 2017 (val); AP: 50.

- [2D Human Pose Estimation on COCO 2017 (val)](<https://www.sota2.com/research/sota/2d-human-pose-estimation-on-coco-2017-val.md>): 2D Human Pose Estimation; COCO 2017 (val); AP: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2208.11356>)

[PDF](<https://arxiv.org/pdf/2208.11356.pdf>)

[Implementation](<https://github.com/ZhangGongjie/IMFA>)
