# Novel Class Discovery in Semantic Segmentation Source: [Novel Class Discovery in Semantic Segmentation]() Collection index: [llms.txt]() Data updated: 2025-10-28T13:56:52.603Z Authors: Yuyang Zhao, Zhun Zhong, Nicu Sebe, Gim Hee Lee Published: 2021-12-03T13:31:59.000Z ## Abstract We introduce a new setting of Novel Class Discovery in Semantic Segmentation (NCDSS), which aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes. In contrast to existing approaches that look at novel class discovery in image classification, we focus on the more challenging semantic segmentation. In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image, which increases the difficulty in using the unlabeled data. To tackle this new setting, we leverage the labeled base data and a saliency model to coarsely cluster novel classes for model training in our basic framework. Additionally, we propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels, further improving the model performance on the novel classes. Our EUMS utilizes an entropy ranking technique and a dynamic reassignment to distill clean labels, thereby making full use of the noisy data via self-supervised learning. We build the NCDSS benchmark on the PASCAL-5$^i$ dataset and COCO-20$^i$ dataset. Extensive experiments demonstrate the feasibility of the basic framework (achieving an average mIoU of 49.81% on PASCAL-5$^i$) and the effectiveness of EUMS framework (outperforming the basic framework by 9.28% mIoU on PASCAL-5$^i$). ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [3D Semantic Segmentation on SemanticPOSS (POSS-3^2)](): 3D Semantic Segmentation; SemanticPOSS (POSS-3^2); Bike IoU: 40.14. - [3D Semantic Segmentation on SemanticPOSS (POSS-3^3)](): 3D Semantic Segmentation; SemanticPOSS (POSS-3^3); IoU (bike): 41.17. - [Novel Class Discovery on SemanticKITTI KITTI-5](): Novel Class Discovery; SemanticKITTI KITTI-5; IoU (bike): 7.53. - [Novel Class Discovery on SemanticKITTI KITTI-5^2](): Novel Class Discovery; SemanticKITTI KITTI-5^2; IoU (bike): 8.26. - [Novel Class Discovery on SemanticKITTI KITTI-4^3](): Novel Class Discovery; SemanticKITTI KITTI-4^3; IoU (bike): 3.95. - [3D Semantic Segmentation on SemanticPOSS (POSS-3^1)](): 3D Semantic Segmentation; SemanticPOSS (POSS-3^1); IoU (Bike): 15.17. - [Novel Class Discovery on SemanticPOSS Split 2](): Novel Class Discovery; SemanticPOSS Split 2; Bike Performance: 40.1. - [Novel Class Discovery on SemanticPOSS (Split 3)](): Novel Class Discovery; SemanticPOSS (Split 3); Bike Accuracy: 41.2. - [3D Semantic Segmentation on SemanticPOSS (POSS-4^0)](): 3D Semantic Segmentation; SemanticPOSS (POSS-4^0); IoU (bike): 25.67. - [Novel Class Discovery on SemanticKITTI KITTI-5^0](): Novel Class Discovery; SemanticKITTI KITTI-5^0; IoU (bike): 5.28. - [Novel Class Discovery on SemanticPOSS (Split 0)](): Novel Class Discovery; SemanticPOSS (Split 0); Bike Performance: 25.7. - [Novel Class Discovery on SemanticPOSS (Split 1)](): Novel Class Discovery; SemanticPOSS (Split 1); Bike Score: 15.2. - [Novel Class Discovery on SemanticKITTI (Split 1)](): Novel Class Discovery; SemanticKITTI (Split 1); Novel Class Score: 24.6. - [Novel Class Discovery on SemanticKITTI (Split 2)](): Novel Class Discovery; SemanticKITTI (Split 2); Novel Class Score: 24.2. - [Novel Class Discovery on SemanticKITTI (Split 3)](): Novel Class Discovery; SemanticKITTI (Split 3); Novel Class Score: 12.4. - [Novel Class Discovery on SemanticKITTI (Split 4)](): Novel Class Discovery; SemanticKITTI (Split 4); Novel Class Score: 7.1. ## Original sources and implementation [Original paper]() [PDF]()