# SCRIB: Set-classifier with Class-specific Risk Bounds for Blackbox Models

Source: [SCRIB: Set-classifier with Class-specific Risk Bounds for Blackbox Models](<https://www.sota2.com/research/paper/68fa696f1e209131cea22099>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:59:51.007Z

Authors: Zhen Lin, Cao Xiao, Lucas Glass, M. Brandon Westover, Jimeng Sun

Published: 2021-03-05T21:06:12.000Z

## Abstract

Despite deep learning (DL) success in classification problems, DL classifiers do not provide a sound mechanism to decide when to refrain from predicting. Recent works tried to control the overall prediction risk with classification with rejection options. However, existing works overlook the different significance of different classes. We introduce Set-classifier with Class-specific RIsk Bounds (SCRIB) to tackle this problem, assigning multiple labels to each example. Given the output of a black-box model on the validation set, SCRIB constructs a set-classifier that controls the class-specific prediction risks with a theoretical guarantee. The key idea is to reject when the set classifier returns more than one label. We validated SCRIB on several medical applications, including sleep staging on electroencephalogram (EEG) data, X-ray COVID image classification, and atrial fibrillation detection based on electrocardiogram (ECG) data. SCRIB obtained desirable class-specific risks, which are 35\\%-88\\% closer to the target risks than baseline methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2103.03945>)

[PDF](<https://arxiv.org/pdf/2103.03945.pdf>)
