# Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles Source: [Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles]() Collection index: [llms.txt]() Data updated: 2025-10-25T23:28:26.804Z Authors: Simon Bultmann, Jan Quenzel, Sven Behnke Published: 2021-08-14T20:16:08.000Z ## Abstract Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. In this work, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from multiple modalities augments 3D point clouds and image segmentation masks while also generating an allocentric semantic map. Our system provides augmented semantic images and point clouds with $\\approx\\,$9$\\,$Hz. We evaluate the integrated system in real-world experiments in an urban environment. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()