A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection

About

Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks. We analyze its failure modes for near-OOD detection and propose a simple fix called relative Mahalanobis distance (RMD) which improves performance and is more robust to hyperparameter choice. On a wide selection of challenging vision, language, and biology OOD benchmarks (CIFAR-100 vs CIFAR-10, CLINC OOD intent detection, Genomics OOD), we show that RMD meaningfully improves upon MD performance (by up to 15% AUROC on genomics OOD).

Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, Balaji Lakshminarayanan · 2021

Related benchmarks

Task	Dataset	Result
Image Classification	ImageNet-1k (val)	Top-1 Accuracy—	1,498
Out-of-Distribution Detection	SUN OOD with ImageNet-1k In-distribution (test)	AUROC87.41	267
Out-of-Distribution Detection	iNaturalist	AUROC96.1	252
Out-of-Distribution Detection	Textures	AUROC0.8938	186
Out-of-Distribution Detection	ImageNet-1k ID iNaturalist OOD	FPR9510.62	143
OOD Detection	ImageNet 1k (test)	FPR9534.6	134
Out-of-Distribution Detection	OpenOOD average of NINCO, iNat, SSB-Hard, OpenImages-O, Textures	FPR @ 95%26.9	130
Out-of-Distribution Detection	OpenImage-O	AUROC92.61	117
Out-of-Distribution Detection	ImageNet	AUROC89.3	113
Out-of-Distribution Detection	ImageNet-1k Textures ID OOD	AUROC89.38	96