# AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation Source: [AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation]() Collection index: [llms.txt]() Data updated: 2026-01-04T05:05:14.611Z Authors: Denis Gudovskiy, Luca Rigazio, Shun Ishizaka, Kazuki Kozuka, Sotaro Tsukizawa Published: 2021-03-10T04:05:33.000Z ## Abstract AutoAugment has sparked an interest in automated augmentation methods for deep learning models. These methods estimate image transformation policies for train data that improve generalization to test data. While recent papers evolved in the direction of decreasing policy search complexity, we show that those methods are not robust when applied to biased and noisy data. To overcome these limitations, we reformulate AutoAugment as a generalized automated dataset optimization (AutoDO) task that minimizes the distribution shift between test data and distorted train dataset. In our AutoDO model, we explicitly estimate a set of per-point hyperparameters to flexibly change distribution of train data. In particular, we include hyperparameters for augmentation, loss weights, and soft-labels that are jointly estimated using implicit differentiation. We develop a theoretical probabilistic interpretation of this framework using Fisher information and show that its complexity scales linearly with the dataset size. Our experiments on SVHN, CIFAR-10/100, and ImageNet classification show up to 9.3% improvement for biased datasets with label noise compared to prior methods and, importantly, up to 36.6% gain for underrepresented SVHN classes. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on CIFAR-100 (test) (IR=1, NR=0.0)](): Image Classification; CIFAR-100 IR=1, NR=0.0 (test); Top-1 Error Rate: 22.2. - [Image Classification on CIFAR-100 IR=10, NR=0.0 (test)](): Image Classification; CIFAR-100 IR=10, NR=0.0 (test); Top-1 Error Rate: 36.7. - [Image Classification on CIFAR-100 (IR=1, NR=0.1) (test)](): Image Classification; CIFAR-100 (IR=1, NR=0.1) (test); Top-1 Error Rate: 27.3. - [Image Classification on CIFAR-100 IR=10, NR=0.1 (test)](): Image Classification; CIFAR-100 IR=10, NR=0.1 (test); Top-1 Error Rate: 39.6. - [Image Classification on CIFAR-10 (test) (IR-NR Robustness Error Rates)](): Image Classification; CIFAR-10 (test); Top-1 Error Rate (IR-NR 1-0.0): 4.9. - [Image Classification on SVHN (test)](): Image Classification; SVHN (test); Accuracy: not recorded. - [Image Classification on ImageNet (test)](): Image Classification; ImageNet (test); Top-1 Acc: not recorded. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()