DooDLeNet: Double DeepLab Enhanced Feature Fusion for Thermal-color Semantic Segmentation

In this paper we present a new approach for feature fusion between RGB and LWIR Thermal images for the task of semantic segmentation for driving perception. We propose DooDLeNet, a double DeepLab architecture with specialized encoder-decoders for thermal and color modalities and a shared decoder for final segmentation. We combine two strategies for feature fusion: confidence weighting and correlation weighting. We report state-of-the-art mean IoU results on the MF dataset.

Oriel Frigo, Lucien Martin-Gaffé, Catherine Wacongne · 2022