# Dynamic Differential-Privacy Preserving SGD Source: [Dynamic Differential-Privacy Preserving SGD]() Collection index: [llms.txt]() Data updated: 2025-10-25T23:52:06.013Z Authors: Jian Du, Song Li, Xiangyi Chen, Siheng Chen, Mingyi Hong Published: 2021-10-30T04:45:11.000Z ## Abstract The vanilla Differentially-Private Stochastic Gradient Descent (DP-SGD), including DP-Adam and other variants, ensures the privacy of training data by uniformly distributing privacy costs across training steps. The equivalent privacy costs controlled by maintaining the same gradient clipping thresholds and noise powers in each step result in unstable updates and a lower model accuracy when compared to the non-DP counterpart. In this paper, we propose the dynamic DP-SGD (along with dynamic DP-Adam, and others) to reduce the performance loss gap while maintaining privacy by dynamically adjusting clipping thresholds and noise powers while adhering to a total privacy budget constraint. Extensive experiments on a variety of deep learning tasks, including image classification, natural language processing, and federated learning, demonstrate that the proposed dynamic DP-SGD algorithm stabilizes updates and, as a result, significantly improves model accuracy in the strong privacy protection region when compared to the vanilla DP-SGD. We also conduct theoretical analysis to better understand the privacy-utility trade-off with dynamic DP-SGD, as well as to learn why Dynamic DP-SGD can outperform vanilla DP-SGD. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Text Classification on GLUE (SST2, QNLI, MNLI subset)](): Text Classification; GLUE; Accuracy (SST-2): 83.4. ## Original sources and implementation [Original paper]() [PDF]()