# Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation Source: [Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation]() Collection index: [llms.txt]() Data updated: 2025-10-25T23:22:37.170Z Authors: Yufei Wang, Haoliang Li, Lap-pui Chau, Alex C. Kot Published: 2021-07-06T14:08:54.000Z ## Abstract Though convolutional neural networks are widely used in different tasks, lack of generalization capability in the absence of sufficient and representative data is one of the challenges that hinder their practical application. In this paper, we propose a simple, effective, and plug-and-play training strategy named Knowledge Distillation for Domain Generalization (KDDG) which is built upon a knowledge distillation framework with the gradient filter as a novel regularization term. We find that both the \`\`richer dark knowledge" from the teacher network, as well as the gradient filter we proposed, can reduce the difficulty of learning the mapping which further improves the generalization ability of the model. We also conduct experiments extensively to show that our framework can significantly improve the generalization capability of deep neural networks in different tasks including image classification, segmentation, reinforcement learning by comparing our method with existing state-of-the-art domain generalization techniques. Last but not the least, we propose to adopt two metrics to analyze our proposed method in order to better understand how our proposed method benefits the generalization capability of deep neural networks. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Optic Disc Segmentation on Fundus Image (held-out target domain)](): Optic Disc Segmentation; Fundus Image (held-out target domain); Dice Coefficient: 91.78. - [Optic Cup Segmentation on Fundus Image (held-out domain)](): Optic Cup Segmentation; Fundus Image (held-out domain); Dice: 79.84. - [Semantic Segmentation on Cityscapes (val) (mIoU only)](): Semantic segmentation; Cityscapes (val); mIoU: 43.5. ## Original sources and implementation [Original paper]() [PDF]()