# Learning to Generate Synthetic Training Data using Gradient Matching and Implicit Differentiation

Source: [Learning to Generate Synthetic Training Data using Gradient Matching and Implicit Differentiation](<https://www.sota2.com/research/paper/68fa6bfc1e209131cea316f2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:01:55.246Z

Authors: Dmitry Medvedev, Alexander D'yakonov

Published: 2022-03-16T11:45:32.000Z

## Abstract

Using huge training datasets can be costly and inconvenient. This article explores various data distillation techniques that can reduce the amount of data required to successfully train deep networks. Inspired by recent ideas, we suggest new data distillation techniques based on generative teaching networks, gradient matching, and the Implicit Function Theorem. Experiments with the MNIST image classification problem show that the new methods are computationally more efficient than previous ones and allow to increase the performance of models trained on distilled data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2203.08559>)

[PDF](<https://arxiv.org/pdf/2203.08559.pdf>)