# Initialization Matters for Adversarial Transfer Learning Source: [Initialization Matters for Adversarial Transfer Learning]() Collection index: [llms.txt]() Data updated: 2026-01-02T13:32:22.896Z Authors: Andong Hua, Jindong Gu, Zhiyu Xue, Nicholas Carlini, Eric Wong, Yao Qin Published: 2023-12-10T00:51:05.000Z ## Abstract With the prevalence of the Pretraining-Finetuning paradigm in transfer learning, the robustness of downstream tasks has become a critical concern. In this work, we delve into adversarial robustness in transfer learning and reveal the critical role of initialization, including both the pretrained model and the linear head. First, we discover the necessity of an adversarially robust pretrained model. Specifically, we reveal that with a standard pretrained model, Parameter-Efficient Finetuning (PEFT) methods either fail to be adversarially robust or continue to exhibit significantly degraded adversarial robustness on downstream tasks, even with adversarial training during finetuning. Leveraging a robust pretrained model, surprisingly, we observe that a simple linear probing can outperform full finetuning and other PEFT methods with random initialization on certain datasets. We further identify that linear probing excels in preserving robustness from the robust pretraining. Based on this, we propose Robust Linear Initialization (RoLI) for adversarial finetuning, which initializes the linear head with the weights obtained by adversarial linear probing to maximally inherit the robustness from pretraining. Across five different image classification datasets, we demonstrate the effectiveness of RoLI and achieve new state-of-the-art results. Our code is available at \\url{https://github.com/DongXzz/RoLI}. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on CIFAR100 (Adversarial Robustness - Clean & PGD)](): Image Classification; CIFAR100; Clean Accuracy: 76.25. - [Adversarial Robustness on CUB200 (Image Classification)](): Image Classification; CUB200; Accuracy (Clean): 74.28. - [Adversarial Robustness Evaluation on Stanford Dogs (Image Classification)](): Image Classification; Stanford Dogs; Clean Accuracy: 92.02. - [Image Classification on CIFAR10 (Clean and PGD Robustness)](): Image Classification; CIFAR10; Clean Accuracy: 94.18. - [Image Classification on Caltech256 (Robustness Metrics)](): Image Classification; Caltech256; Accuracy (Clean): 83.78. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()