# RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Clutter Source: [RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Clutter]() Collection index: [llms.txt]() Data updated: 2025-10-25T21:13:58.604Z Authors: Max Schwarz, Anton Milan, Arul Selvam Periyasamy, Sven Behnke Published: 2018-10-01T16:55:53.000Z ## Abstract Autonomous robotic manipulation in clutter is challenging. A large variety of objects must be perceived in complex scenes, where they are partially occluded and embedded among many distractors, often in restricted spaces. To tackle these challenges, we developed a deep-learning approach that combines object detection and semantic segmentation. The manipulation scenes are captured with RGB-D cameras, for which we developed a depth fusion method. Employing pretrained features makes learning from small annotated robotic data sets possible. We evaluate our approach on two challenging data sets: one captured for the Amazon Picking Challenge 2016, where our team NimbRo came in second in the Stowing and third in the Picking task, and one captured in disaster-response scenarios. The experiments show that object detection and semantic segmentation complement each other and can be combined to yield reliable object perception. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()