# In Automation We Trust: Investigating the Role of Uncertainty in Active Learning Systems

Source: [In Automation We Trust: Investigating the Role of Uncertainty in Active Learning Systems](<https://www.sota2.com/research/paper/68fa67561e209131cea159fd>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:12:13.696Z

Authors: Michael L. Iuzzolino, Tetsumichi Umada, Nisar R. Ahmed, Danielle A. Szafir

Published: 2020-04-02T00:52:49.000Z

## Abstract

We investigate how different active learning (AL) query policies coupled with classification uncertainty visualizations affect analyst trust in automated classification systems. A current standard policy for AL is to query the oracle (e.g., the analyst) to refine labels for datapoints where the classifier has the highest uncertainty. This is an optimal policy for the automation system as it yields maximal information gain. However, model-centric policies neglect the effects of this uncertainty on the human component of the system and the consequent manner in which the human will interact with the system post-training. In this paper, we present an empirical study evaluating how AL query policies and visualizations lending transparency to classification influence trust in automated classification of image data. We found that query policy significantly influences an analyst's trust in an image classification system, and we use these results to propose a set of oracle query policies and visualizations for use during AL training phases that can influence analyst trust in classification.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2004.00762>)

[PDF](<https://arxiv.org/pdf/2004.00762.pdf>)
