# Machine Learning with a Reject Option: A survey

Source: [Machine Learning with a Reject Option: A survey](<https://www.sota2.com/research/paper/68fa72471e209131cea557f2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:22:12.513Z

Authors: Kilian Hendrickx, Lorenzo Perini, Dries Van der Plas, Wannes Meert, Jesse Davis

Published: 2021-07-23T14:43:56.000Z

## Abstract

Machine learning models always make a prediction, even when it is likely to be inaccurate. This behavior should be avoided in many decision support applications, where mistakes can have severe consequences. Albeit already studied in 1970, machine learning with rejection recently gained interest. This machine learning subfield enables machine learning models to abstain from making a prediction when likely to make a mistake.   This survey aims to provide an overview on machine learning with rejection. We introduce the conditions leading to two types of rejection, ambiguity and novelty rejection, which we carefully formalize. Moreover, we review and categorize strategies to evaluate a model's predictive and rejective quality. Additionally, we define the existing architectures for models with rejection and describe the standard techniques for learning such models. Finally, we provide examples of relevant application domains and show how machine learning with rejection relates to other machine learning research areas.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Explanation quality rejection on iknl](<https://www.sota2.com/research/sota/explanation-quality-rejection-on-iknl.md>): Explanation quality rejection; iknl; AUROC: 62.

- [Explanation quality rejection on celeba](<https://www.sota2.com/research/sota/explanation-quality-rejection-on-celeba.md>): Explanation quality rejection; CelebA; AUROC: 0.69.

- [Explanation Quality Assessment on Soccer xG Explanations (User Study)](<https://www.sota2.com/research/sota/explanation-quality-assessment-on-soccer-xg-explanations-user-study.md>): Explanation Quality Assessment; Soccer xG Explanations (User Study); AUROC: 0.46.

- [Explanation quality rejection on adult](<https://www.sota2.com/research/sota/explanation-quality-rejection-on-adult.md>): Explanation quality rejection; Adult; AUROC: 0.58.

- [Explanation Quality Rejection on sleepstage](<https://www.sota2.com/research/sota/explanation-quality-rejection-on-sleepstage.md>): Explanation quality rejection; sleepstage; AUROC: 24.

- [Explanation Quality Rejection on Compas (AUROC)](<https://www.sota2.com/research/sota/explanation-quality-rejection-on-compas-auroc.md>): Explanation quality rejection; COMPAS; AUROC: 0.56.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2107.11277>)

[PDF](<https://arxiv.org/pdf/2107.11277.pdf>)
