# Towards Autonomous Driving: a Multi-Modal 360$^{\\circ}$ Perception Proposal

Source: [Towards Autonomous Driving: a Multi-Modal 360$^{\\circ}$ Perception Proposal](<https://www.sota2.com/research/paper/68fa68361e209131cea1acf0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:32:00.581Z

Authors: Jorge Beltr\\'an, Carlos Guindel, Irene Cort\\'es, Alejandro Barrera, Armando Astudillo, Jes\\'us Urdiales, Mario \\'Alvarez, Farid Bekka, Vicente Milan\\'es, Fernando Garc\\'ia

Published: 2020-08-21T20:36:21.000Z

## Abstract

In this paper, a multi-modal 360$^{\\circ}$ framework for 3D object detection and tracking for autonomous vehicles is presented. The process is divided into four main stages. First, images are fed into a CNN network to obtain instance segmentation of the surrounding road participants. Second, LiDAR-to-image association is performed for the estimated mask proposals. Then, the isolated points of every object are processed by a PointNet ensemble to compute their corresponding 3D bounding boxes and poses. Lastly, a tracking stage based on Unscented Kalman Filter is used to track the agents along time. The solution, based on a novel sensor fusion configuration, provides accurate and reliable road environment detection. A wide variety of tests of the system, deployed in an autonomous vehicle, have successfully assessed the suitability of the proposed perception stack in a real autonomous driving application.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2008.09672>)

[PDF](<https://arxiv.org/pdf/2008.09672.pdf>)
