# Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting

Source: [Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting](<https://www.sota2.com/research/paper/68fa70d51e209131cea4d842>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:07:19.331Z

Authors: Chengyu Dong, Liyuan Liu, Jingbo Shang

Published: 2021-10-07T01:15:06.000Z

## Abstract

We show that label noise exists in adversarial training. Such label noise is due to the mismatch between the true label distribution of adversarial examples and the label inherited from clean examples - the true label distribution is distorted by the adversarial perturbation, but is neglected by the common practice that inherits labels from clean examples. Recognizing label noise sheds insights on the prevalence of robust overfitting in adversarial training, and explains its intriguing dependence on perturbation radius and data quality. Also, our label noise perspective aligns well with our observations of the epoch-wise double descent in adversarial training. Guided by our analyses, we proposed a method to automatically calibrate the label to address the label noise and robust overfitting. Our method achieves consistent performance improvements across various models and datasets without introducing new hyper-parameters or additional tuning.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-100 (test) (Tracking Robustness Delta)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-test-tracking-robustness-delta.md>): Image Classification; CIFAR-100 (test); Robust Accuracy (Best): 26.36.

- [Adversarial Robust Image Classification on CIFAR-10 (test)](<https://www.sota2.com/research/sota/adversarial-robust-image-classification-on-cifar-10-test.md>): Image Classification; CIFAR-10 (test); Accuracy (Clean): not recorded.

- [Robustness Accuracy Evaluation on Tiny-ImageNet (test)](<https://www.sota2.com/research/sota/robustness-accuracy-evaluation-on-tiny-imagenet-test.md>): Image Classification; Tiny ImageNet (test); Standard Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2110.03135>)

[PDF](<https://arxiv.org/pdf/2110.03135.pdf>)
