A Dynamical System Perspective for Lipschitz Neural Networks

About

The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the {\em Convex Potential Layer} (CPL). A comprehensive set of experiments on several datasets demonstrates the scalability of our architecture and the benefits as an $\ell_2$-provable defense against adversarial examples.

Laurent Meunier, Blaise Delattre, Alexandre Araujo, Alexandre Allauzen · 2021

Related benchmarks

Task	Dataset	Result
Certified Accuracy	Mean Certified Accuracy 255/255	Total Samples (N)121	15
Certified Accuracy	Mean Certified Accuracy (72/255) (test)	Wins A19	15
Certified Robustness	Mean Certified Accuracy 108/255	Wins A15	15
Certified Robustness	Mean Certified Accuracy 36/255	Wins A16	15
Mean Accuracy	121 benchmarks tasks	Wins A27	15
Image Classification	CIFAR10 (test)	Accuracy0.785	11
Image Classification	CIFAR-100 (test)	Clean Accuracy0.478	11
Image Classification	CIFAR-10 (test)	Clean Accuracy78.5	10
Image Classification	CIFAR-10 (test)	Clean Accuracy78.5	10
Image Classification	CIFAR100 (test)	Natural Accuracy0.478	9