# Do We Need All the Synthetic Data? Targeted Image Augmentation via Diffusion Models

Source: [Do We Need All the Synthetic Data? Targeted Image Augmentation via Diffusion Models](<https://www.sota2.com/research/paper/68fa7be51e209131cea8b264>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-06T05:00:12.548Z

Authors: Dang Nguyen, Jiping Li, Jinghao Zheng, Baharan Mirzasoleiman

Published: 2025-05-27T07:27:03.000Z

## Abstract

Synthetically augmenting training datasets with diffusion models has become an effective strategy for improving the generalization of image classifiers. However, existing approaches typically increase dataset size by 10-30x and struggle to ensure generation diversity, leading to substantial computational overhead. In this work, we introduce TADA (TArgeted Diffusion Augmentation), a principled framework that selectively augments examples that are not learned early in training using faithful synthetic images that preserve semantic features while varying noise. We show that augmenting only this targeted subset consistently outperforms augmenting the entire dataset. Through theoretical analysis on a two-layer CNN, we prove that TADA improves generalization by promoting homogeneity in feature learning speed without amplifying noise. Extensive experiments demonstrate that by augmenting only 30-40% of the training data, TADA improves generalization by up to 2.8% across diverse architectures including ResNet, ViT, ConvNeXt, and Swin Transformer on CIFAR-10/100, TinyImageNet, and ImageNet, using optimizers such as SGD and SAM. Notably, TADA combined with SGD outperforms the state-of-the-art optimizer SAM on CIFAR-100 and TinyImageNet. Furthermore, TADA shows promising improvements on object detection benchmarks, demonstrating its applicability beyond image classification. Our code is available at https://github.com/BigML-CS-UCLA/TADA.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet (test) (Standard Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-test-standard-accuracy.md>): Image Classification; ImageNet (test); Top-1 Accuracy: 74.77.

- [Image Classification on FGVC-Aircraft (test)](<https://www.sota2.com/research/sota/image-classification-on-fgvc-aircraft-test.md>): Image Classification; FGVC-Aircraft (test); Accuracy: not recorded.

- [Image Classification on Stanford Cars (test)](<https://www.sota2.com/research/sota/image-classification-on-stanford-cars-test.md>): Image Classification; Stanford Cars (test); Accuracy: not recorded.

- [Object Detection on MS-COCO 2017 (val)](<https://www.sota2.com/research/sota/object-detection-on-ms-coco-2017-val.md>): Object Detection; MS-COCO 2017 (val); mAP: not recorded.

- [Image Classification on Flowers-102 (test)](<https://www.sota2.com/research/sota/image-classification-on-flowers-102-test.md>): Image Classification; Flowers-102 (test); Top-1 Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2505.21574>)

[PDF](<https://arxiv.org/pdf/2505.21574.pdf>)
