SelectiveNet: A Deep Neural Network with an Integrated Reject Option

About

We consider the problem of selective prediction (also known as reject option) in deep neural networks, and introduce SelectiveNet, a deep neural architecture with an integrated reject option. Existing rejection mechanisms are based mostly on a threshold over the prediction confidence of a pre-trained network. In contrast, SelectiveNet is trained to optimize both classification (or regression) and rejection simultaneously, end-to-end. The result is a deep neural network that is optimized over the covered domain. In our experiments, we show a consistently improved risk-coverage trade-off over several well-known classification and regression datasets, thus reaching new state-of-the-art results for deep selective classification.

Yonatan Geifman, Ran El-Yaniv · 2019

Related benchmarks

Task	Dataset	Result
Selective Classification	CIFAR-10 (test)	Accuracy—	98
Selective Prediction	ImageNet-100	Selective Prediction Error0.2	60
Selective Prediction	Stanford Cars	Selective Prediction Error6.03	60
Selective Prediction	CIFAR-100	Selective Prediction Error1.1	60
Selective Classification	CIFAR-100 (test)	AUC0.744	32
Adversarial Detection	CIFAR-10 clean (test)	TPR-9588.3	23
Adversarial Attack Detection	CIFAR-100 PGD-10 (l_inf, 16/255) 1.0	TPR-9522.02	16
Adversarial Attack Detection	CIFAR-100 PGD-10 (l_inf, 8/255) 1.0	TPR-9536.14	16
Adversarial Attack Detection	CIFAR-100 1.0 (Clean)	TPR-9564.09	16
Adversarial Attack Detection	CIFAR-100 PGD-10 (l_2, 128/255) 1.0	TPR-950.4432	16