# When Do Fewer Coordinates Suffice in DP-SGD?

Source: [When Do Fewer Coordinates Suffice in DP-SGD?](<https://www.sota2.com/research/paper/6a2106d2c10484bac23f0470>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-05T05:00:04.417Z

Authors: Huiqi Zhang, Fang Xie

Published: 2026-06-03T02:46:25.000Z

## Abstract

Differentially private stochastic gradient descent (DP-SGD) injects noise into every updated coordinate, making the injected noise energy scale with the ambient parameter dimension \\(d\\). We ask when private training can update fewer coordinates without losing the signal needed for optimization. We propose \\textsc{TP-TopK} (Two-Phase TopK DP-SGD), a two-phase method for coordinate-sparse private training without public data, in which a private warm-up phase identifies a coordinate support used to guide the main training phase. We give a criterion characterizing when coordinate restriction can be beneficial, show via a nonconvex stationarity bound that under this condition the relevant noise term scales with the active dimension \\(k\\) rather than the full parameter dimension \\(d\\), and provide a lower bound on the reliability of warm-up-based coordinate ranking. Experiments on MNIST, FMNIST, and CIFAR-10 show that learned coordinate supports can retain more gradient energy than size-matched random supports, with the largest gains when the active dimension is small and warm-up scores are informative.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on MNIST (test) (Epsilon-based Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-mnist-test-epsilon-based-accuracy.md>): Image Classification; MNIST (test); Accuracy (\\epsilon=1): 97.79.

- [Image Classification on FMNIST (test) (Epsilon-Weighted Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-fmnist-test-epsilon-weighted-accuracy.md>): Image Classification; fMNIST (test); Accuracy (epsilon=1): 85.28.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.04375>)

[PDF](<https://arxiv.org/pdf/2606.04375.pdf>)