# Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generation Source: [Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generation]() Collection index: [llms.txt]() Data updated: 2025-10-26T03:02:53.994Z Authors: Nina Weng, Paraskevas Pegios, Eike Petersen, Aasa Feragen, Siavash Bigdeli Published: 2023-12-21T15:05:12.000Z ## Abstract Shortcut learning is when a model -- e.g. a cardiac disease classifier -- exploits correlations between the target label and a spurious shortcut feature, e.g. a pacemaker, to predict the target label based on the shortcut rather than real discriminative features. This is common in medical imaging, where treatment and clinical annotations correlate with disease labels, making them easy shortcuts to predict disease. We propose a novel detection and quantification of the impact of potential shortcut features via a fast diffusion-based counterfactual image generation that can synthetically remove or add shortcuts. Via a novel inpainting-based modification we spatially limit the changes made with no extra inference step, encouraging the removal of spatially constrained shortcut features while ensuring that the shortcut-free counterfactuals preserve their remaining image features to a high degree. Using these, we assess how shortcut features influence model predictions. This is enabled by our second contribution: An efficient diffusion-based counterfactual explanation method with significant inference speed-up at comparable image quality as state-of-the-art. We confirm this on two large chest X-ray datasets, a skin lesion dataset, and CelebA. Our code is publicly available at fastdime.compute.dtu.dk. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Counterfactual Explanation on CelebA Blond (Male)](): Counterfactual Explanation; CelebA Blond (Male); NA Score: 68.46. - [Counterfactual Explanation on Camelyon Hosp. 17](): Counterfactual Explanation; Camelyon Hosp. 17; NA: 36. - [Visual Counterfactual Explanation (Age) on CelebA Standard](): Visual Counterfactual Explanation (Age); CelebA Standard; FID: 3.6. - [Counterfactual Explanation on Celeb-A Blond](): Counterfactual Explanation; Celeb-A Blond; NA Score: 62.5. - [Counterfactual Explanation on Celeb-A Smile](): Counterfactual Explanation; Celeb-A Smile; Performance Score: 78. - [Visual Counterfactual Explanation (Smile) on CelebA Standard](): Visual Counterfactual Explanation (Smile); CelebA Standard; FID: 3.24. - [Counterfactual Explanation on ImageNet Zebra - Sorrel](): Counterfactual Explanation; ImageNet Zebra - Sorrel; FID: 96.5. - [Counterfactual Explanation on ImageNet (Cheetah - Cougar)](): Counterfactual Explanation; ImageNet (Cheetah - Cougar); FID: 133. - [Counterfactual Explanation on ImageNet Egyptian Cat - Persian Cat](): Counterfactual Explanation; ImageNet Egyptian Cat - Persian Cat; FID: 193.6. ## Original sources and implementation [Original paper]() [PDF]()