# Self-Supervised Pretraining for Differentially Private Learning

Source: [Self-Supervised Pretraining for Differentially Private Learning](<https://www.sota2.com/research/paper/68fa6d341e209131cea3861a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:28:36.143Z

Authors: Arash Asadian, Evan Weidner, Lei Jiang

Published: 2022-06-14T19:30:23.000Z

## Abstract

We demonstrate self-supervised pretraining (SSP) is a scalable solution to deep learning with differential privacy (DP) regardless of the size of available public datasets in image classification. When facing the lack of public datasets, we show the features generated by SSP on only one single image enable a private classifier to obtain much better utility than the non-learned handcrafted features under the same privacy budget. When a moderate or large size public dataset is available, the features produced by SSP greatly outperform the features trained with labels on various complex private datasets under the same private budget. We also compared multiple DP-enabled training frameworks to train a private classifier on the features generated by SSP. Finally, we report a non-trivial utility 25.3\\% of a private ImageNet-1K dataset when $\\epsilon=3$. Our source code can be found at \\url{https://github.com/UnchartedRLab/SSP}.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2206.07125>)

[PDF](<https://arxiv.org/pdf/2206.07125.pdf>)