Learning Counterfactually Invariant Predictors

About

Notions of counterfactual invariance (CI) have proven essential for predictors that are fair, robust, and generalizable in the real world. We propose graphical criteria that yield a sufficient condition for a predictor to be counterfactually invariant in terms of a conditional independence in the observational distribution. In order to learn such predictors, we propose a model-agnostic framework, called Counterfactually Invariant Prediction (CIP), building on the Hilbert-Schmidt Conditional Independence Criterion (HSCIC), a kernel-based conditional dependence measure. Our experimental results demonstrate the effectiveness of CIP in enforcing counterfactual invariance across various simulated and real-world datasets including scalar and multi-variate settings.

Francesco Quinzan, Cecilia Casolo, Krikamol Muandet, Yucen Luo, Niki Kilbertus · 2022

Related benchmarks

Task	Dataset	Result
Classification	Waterbirds bias-free (test)	Balanced Accuracy85.3	11
Classification	FairFace bias-free (test)	Balanced Accuracy74.8	11
Classification	MNLI bias-free (test)	Worst Group Accuracy71.6	10
Classification	YaleB bias-free (test)	Balanced Accuracy70.8	7
Regression	dSprites bias-free (test)	R20.568	7
Regression	Blob bias-free (test)	R2 Score0.47	7