# Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond

Source: [Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond](<https://www.sota2.com/research/paper/68fa69b41e209131cea239d3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:06:39.153Z

Authors: Negin Majidi, Ehsan Amid, Hossein Talebi, Manfred K. Warmuth

Published: 2021-04-03T22:54:49.000Z

## Abstract

Many learning tasks in machine learning can be viewed as taking a gradient step towards minimizing the average loss of a batch of examples in each training iteration. When noise is prevalent in the data, this uniform treatment of examples can lead to overfitting to noisy examples with larger loss values and result in poor generalization. Inspired by the expert setting in on-line learning, we present a flexible approach to learning from noisy examples. Specifically, we treat each training example as an expert and maintain a distribution over all examples. We alternate between updating the parameters of the model using gradient descent and updating the example weights using the exponentiated gradient update. Unlike other related methods, our approach handles a general class of loss functions and can be applied to a wide range of noise types and applications. We show the efficacy of our approach for multiple learning settings, namely noisy principal component analysis and a variety of noisy classification problems.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-100 (test) (Symmetric/Asymmetric Robustness)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-test-symmetric-asymmetric-robustness.md>): Image Classification; CIFAR100 (test); Top-1 Acc (Symmetric, 0.2): 69.53.

- [Image Classification on CIFAR10 (test) (Symmetric/Asymmetric Robustness)](<https://www.sota2.com/research/sota/image-classification-on-cifar10-test-symmetric-asymmetric-robustness.md>): Image Classification; CIFAR10 (test); Top-1 Acc (Sym, 0.2): 90.25.

- [Image Classification on Clothing1M (test)](<https://www.sota2.com/research/sota/image-classification-on-clothing1m-test.md>): Image Classification; Clothing1M (test); Accuracy: 69.42.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2104.01493>)

[PDF](<https://arxiv.org/pdf/2104.01493.pdf>)