# Bridging the View Disparity Between Radar and Camera Features for Multi-modal Fusion 3D Object Detection

Source: [Bridging the View Disparity Between Radar and Camera Features for Multi-modal Fusion 3D Object Detection](<https://www.sota2.com/research/paper/68fa6dc81e209131cea3c2f9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:44:11.594Z

Authors: Taohua Zhou, Yining Shi, Junjie Chen, Kun Jiang, Mengmeng Yang, Diange Yang

Published: 2022-08-25T13:21:37.000Z

## Abstract

Environmental perception with the multi-modal fusion of radar and camera is crucial in autonomous driving to increase accuracy, completeness, and robustness. This paper focuses on utilizing millimeter-wave (MMW) radar and camera sensor fusion for 3D object detection. A novel method that realizes the feature-level fusion under the bird's-eye view (BEV) for a better feature representation is proposed. Firstly, radar points are augmented with temporal accumulation and sent to a spatial-temporal encoder for radar feature extraction. Meanwhile, multi-scale image 2D features which adapt to various spatial scales are obtained by image backbone and neck model. Then, image features are transformed to BEV with the designed view transformer. In addition, this work fuses the multi-modal features with a two-stage fusion model called point-fusion and ROI-fusion, respectively. Finally, a detection head regresses objects category and 3D locations. Experimental results demonstrate that the proposed method realizes the state-of-the-art (SOTA) performance under the most crucial detection metrics-mean average precision (mAP) and nuScenes detection score (NDS) on the challenging nuScenes dataset.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [3D Object Detection on nuScenes v1.0 (test)](<https://www.sota2.com/research/sota/3d-object-detection-on-nuscenes-v1-0-test.md>): 3D Object Detection; NuScenes v1.0 (test); mAP: 40.6.

- [3D Object Detection on nuScenes (val)](<https://www.sota2.com/research/sota/3d-object-detection-on-nuscenes-val.md>): 3D Object Detection; nuScenes (val); NDS: 49.7.

- [3D Object Detection on nuScenes (test)](<https://www.sota2.com/research/sota/3d-object-detection-on-nuscenes-test.md>): 3D Object Detection; nuScenes (test); mAP: 40.6.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2208.12079>)

[PDF](<https://arxiv.org/pdf/2208.12079.pdf>)