Chordal Sparsity for Lipschitz Constant Estimation of Deep Neural Networks

Lipschitz constants of neural networks allow for guarantees of robustness in image classification, safety in controller design, and generalizability beyond the training data. As calculating Lipschitz constants is NP-hard, techniques for estimating Lipschitz constants must navigate the trade-off betw...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2022-04
Main Authors: Xue, Anton, Lindemann, Lars, Robey, Alexander, Hassani, Hamed, Pappas, George J, Alur, Rajeev
Format: Article
Language:English
Subjects:
Artificial neural networks
Computing costs
Control systems design
Estimation
Image classification
Neural networks
Robustness (mathematics)
Semidefinite programming
Sparsity
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Lipschitz constants of neural networks allow for guarantees of robustness in image classification, safety in controller design, and generalizability beyond the training data. As calculating Lipschitz constants is NP-hard, techniques for estimating Lipschitz constants must navigate the trade-off between scalability and accuracy. In this work, we significantly push the scalability frontier of a semidefinite programming technique known as LipSDP while achieving zero accuracy loss. We first show that LipSDP has chordal sparsity, which allows us to derive a chordally sparse formulation that we call Chordal-LipSDP. The key benefit is that the main computational bottleneck of LipSDP, a large semidefinite constraint, is now decomposed into an equivalent collection of smaller ones: allowing Chordal-LipSDP to outperform LipSDP particularly as the network depth grows. Moreover, our formulation uses a tunable sparsity parameter that enables one to gain tighter estimates without incurring a significant computational cost. We illustrate the scalability of our approach through extensive numerical experiments.
ISSN:2331-8422