# DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models Source: [DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models]() Collection index: [llms.txt]() Data updated: 2026-05-13T05:00:20.520Z Authors: Haaris Mehmood, Jie Xu, Karthikeyan Saravanan, Rogier Van Dalen, Mete Ozay Published: 2026-05-11T09:32:21.000Z ## Abstract Federated learning (FL) enables the collaborative training of large-scale language models (LLMs) across edge devices while keeping user data on-device. However, FL still exposes sensitive information through client-provided gradients. Differentially private stochastic gradient descent (DP-SGD) mitigates this risk by clipping each client's contribution to a threshold $C$ and adding noise proportional to $C$. Existing adaptive clipping techniques dynamically adjust $C$ but demand tedious hyperparameter tuning, which can erode the privacy budget. In this paper, we introduce DP-LAC, a method that first estimates an initial clipping threshold within an order of magnitude of the optimum using private histogram estimation, and then adapts this threshold during training without consuming additional privacy budget or introducing new hyperparameters. Empirical results show that DP-LAC outperforms both state-of-the-art adaptive clipping methods and vanilla DP-SGD, achieving an average accuracy gain of $6.6\\%$. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Text Classification on GLUE (SST2, QNLI, MNLI subset)](): Text Classification; GLUE; Accuracy (SST-2): 89.1. ## Original sources and implementation [Original paper]() [PDF]()