LiRaFusion: Deep Adaptive LiDAR-Radar Fusion for 3D Object Detection

About

We propose LiRaFusion to tackle LiDAR-radar fusion for 3D object detection to fill the performance gap of existing LiDAR-radar detectors. To improve the feature extraction capabilities from these two modalities, we design an early fusion module for joint voxel feature encoding, and a middle fusion module to adaptively fuse feature maps via a gated network. We perform extensive evaluation on nuScenes to demonstrate that LiRaFusion leverages the complementary information of LiDAR and radar effectively and achieves notable improvement over existing methods.

Jingyu Song, Lingjun Zhao, Katherine A. Skinner · 2024

Related benchmarks

Task	Dataset	Result
3D Object Detection	VoD Clear Weather, Long Distance > 30m (test)	mAP (Car)62.46	6
3D Object Detection	VoD Heavy Fog, Long Distance > 30m (test)	3D AP (Car)21.3	6
3D Object Detection	VoD Clear Weather, Short Distance <= 30m (test)	mAP (Car)70.82	6
3D Object Detection	VoD Heavy Fog, Short Distance <= 30m (test)	3D AP (Car)33.5	6
3D Object Detection	VoD-Fog Mild Fog	AP Car Easy (IoU=0.5)78.3	6
3D Object Detection	VoD-Fog W/o Fog	AP (Car, Easy, IoU=0.5)84.4	6
3D Object Detection	VoD-Fog Heavy	AP (Car Easy, IoU=0.5)66.5	6