# Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning

Source: [Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning](<https://www.sota2.com/research/paper/68fa75241e209131cea661fd>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-08-05T05:00:12.726Z

Authors: Hengxiang Zhang, Qiang Hu, Hongxin Wei

Published: 2024-10-09T12:13:49.000Z

## Abstract

Over-parameterized models are typically vulnerable to membership inference attacks, which aim to determine whether a specific sample is included in the training of a given model. Previous Weight regularizations (e.g., L1 regularization) typically impose uniform penalties on all parameters, leading to a suboptimal tradeoff between model utility and privacy. In this work, we first show that only a small fraction of parameters substantially impact the privacy risk. In light of this, we propose Privacy-aware Sparsity Tuning (PAST), a simple fix to the L1 Regularization, by employing adaptive penalties to different parameters. Our key idea behind PAST is to promote sparsity in parameters that significantly contribute to privacy leakage. In particular, we construct the adaptive weight for each parameter based on its privacy sensitivity, i.e., the gradient of the loss gap with respect to the parameter. Using PAST, the network shrinks the loss gap between members and non-members, leading to strong resistance to privacy attacks. Extensive experiments demonstrate the superiority of PAST, achieving a state-of-the-art balance in the privacy-utility trade-off.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-10 original (test) (LiRA and RMIA Metrics)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-original-test-lira-and-rmia-metrics.md>): Image Classification; CIFAR-10 original (test); Accuracy (Test): 54.84.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2410.06814>)

[PDF](<https://arxiv.org/pdf/2410.06814.pdf>)