# Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy Source: [Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy]() Collection index: [llms.txt]() Data updated: 2026-03-07T05:00:04.447Z Authors: Rustem Islamov, Samuel Horvath, Aurelien Lucchi, Peter Richtarik, Eduard Gorbunov Published: 2025-02-17T11:16:21.000Z ## Abstract Strong Differential Privacy (DP) and Optimization guarantees are two desirable properties for a method in Federated Learning (FL). However, existing algorithms do not achieve both properties at once: they either have optimal DP guarantees but rely on restrictive assumptions such as bounded gradients/bounded data heterogeneity, or they ensure strong optimization performance but lack DP guarantees. To address this gap in the literature, we propose and analyze a new method called Clip21-SGD2M based on a novel combination of clipping, heavy-ball momentum, and Error Feedback. In particular, for non-convex smooth distributed problems with clients having arbitrarily heterogeneous data, we prove that Clip21-SGD2M has optimal convergence rate and also near optimal (local-)DP neighborhood. Our numerical experiments on non-convex logistic regression and training of neural networks highlight the superiority of Clip21-SGD2M over baselines in terms of the optimization performance for a given DP-budget. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on MNIST (test) (Epsilon-based Robustness Evaluation)](): Image Classification; MNIST (test); Accuracy (epsilon=3): 62.6. ## Original sources and implementation [Original paper]() [PDF]()