# On the Price of Privacy for Language Identification and Generation

Source: [On the Price of Privacy for Language Identification and Generation](<https://www.sota2.com/research/paper/69d7329b7d0172140ec76bd6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-10T05:00:13.311Z

Authors: Xiaoyu Li, Andi Han, Jiaojiao Jiang, Junbin Gao

Published: 2026-04-08T16:04:16.000Z

## Abstract

As large language models (LLMs) are increasingly trained on sensitive user data, understanding the fundamental cost of privacy in language learning becomes essential. We initiate the study of differentially private (DP) language identification and generation in the agnostic statistical setting, establishing algorithms and matching lower bounds that precisely quantify the cost of privacy. For both tasks, approximate $(\\varepsilon, \\delta)$-DP with constant $\\varepsilon \> 0$ recovers the non-private error rates: $\\exp(-r(n))$ for identification (for any $r(n) = o(n)$) and $\\exp(-\\Omega(n))$ for generation. Under pure $\\varepsilon$-DP, the exponents degrade by a multiplicative factor of $\\min\\{1, \\varepsilon\\}$, which we show is tight up to constants. Notably, for generation under pure DP with mild assumptions, the upper bound $\\exp(-\\min\\{1,\\varepsilon\\} \\cdot \\Omega(n))$ matches the lower bound up to some constants, establishing an optimal rate. Our results show that the cost of privacy in language learning is surprisingly mild: absent entirely under approximate DP, and exactly a $\\min\\{1,\\varepsilon\\}$ factor in the exponent under pure DP.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Language Generation on Agnostic Statistical Language Learning](<https://www.sota2.com/research/sota/language-generation-on-agnostic-statistical-language-learning.md>): Language Generation; Agnostic Statistical Language Learning; Error Rate: 1.

- [Language Identification on Agnostic Statistical Language Learning](<https://www.sota2.com/research/sota/language-identification-on-agnostic-statistical-language-learning.md>): Language Identification; Agnostic Statistical Language Learning; Error Rate: 1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.07238>)

[PDF](<https://arxiv.org/pdf/2604.07238.pdf>)