# Automated Discovery of Adaptive Attacks on Adversarial Defenses

Source: [Automated Discovery of Adaptive Attacks on Adversarial Defenses](<https://www.sota2.com/research/paper/68fa6b161e209131cea2c0c7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-24T16:09:17.297Z

Authors: Chengyuan Yao, Pavol Bielik, Petar Tsankov, Martin Vechev

Published: 2021-02-23T18:43:24.000Z

## Abstract

Reliable evaluation of adversarial defenses is a challenging task, currently limited to an expert who manually crafts attacks that exploit the defense's inner workings or approaches based on an ensemble of fixed attacks, none of which may be effective for the specific defense at hand. Our key observation is that adaptive attacks are composed of reusable building blocks that can be formalized in a search space and used to automatically discover attacks for unknown defenses. We evaluated our approach on 24 adversarial defenses and show that it outperforms AutoAttack, the current state-of-the-art tool for reliable evaluation of adversarial defenses: our tool discovered significantly stronger attacks by producing 3.0\\%-50.8\\% additional adversarial examples for 10 models, while obtaining attacks with slightly stronger or similar strength for the remaining models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Adversarial Robustness on CIFAR-10 (L2, eps=8/255)](<https://www.sota2.com/research/sota/adversarial-robustness-on-cifar-10-l2-eps-8-255.md>): Adversarial Robustness; CIFAR-10 L2, epsilon=8/255; Robust Acc (AA): not recorded.

- [Adversarial Robustness on CIFAR-10 (L-inf, eps=4/255)](<https://www.sota2.com/research/sota/adversarial-robustness-on-cifar-10-l-inf-eps-4-255.md>): Adversarial Robustness; CIFAR-10 L-infinity, epsilon=4/255; Robust Accuracy (AA): not recorded.

- [Adversarial Robustness on CIFAR-10 (L-infinity, epsilon=8/255)](<https://www.sota2.com/research/sota/adversarial-robustness-on-cifar-10-l-infinity-epsilon-8-255.md>): Adversarial Robustness; CIFAR-10 L-infinity, epsilon=8/255; Robust Acc (AA): not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2102.11860>)

[PDF](<https://arxiv.org/pdf/2102.11860.pdf>)

[Implementation](<https://github.com/eth-sri/adaptive-auto-attack>)