# Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications

Source: [Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications](<https://www.sota2.com/research/paper/68fa66921e209131cea10fb0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:53:21.106Z

Authors: Nicholas Carlini, \\'Ulfar Erlingsson, Nicolas Papernot

Published: 2019-10-29T17:44:35.000Z

## Abstract

We develop techniques to quantify the degree to which a given (training or testing) example is an outlier in the underlying distribution. We evaluate five methods to score examples in a dataset by how well-represented the examples are, for different plausible definitions of "well-represented", and apply these to four common datasets: MNIST, Fashion-MNIST, CIFAR-10, and ImageNet. Despite being independent approaches, we find all five are highly correlated, suggesting that the notion of being well-represented can be quantified. Among other uses, we find these methods can be combined to identify (a) prototypical examples (that match human expectations); (b) memorized training examples; and, (c) uncommon submodes of the dataset. Further, we show how we can utilize our metrics to determine an improved ordering for curriculum learning, and impact adversarial robustness. We release all metric values on training and test sets we studied.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1910.13427>)

[PDF](<https://arxiv.org/pdf/1910.13427.pdf>)