Deep residual networks (ResNets) have demonstrated outstanding success in computer vision tasks, attributed to their ability to maintain gradient flow through deep architectures. Simultaneously, controlling the Lipschitz constant in neural networks has emerged as an essential area of research to enhance adversarial robustness and network certifiability. This paper presents a rigorous approach to the general design of $\mathcal{L}$-Lipschitz deep residual networks using a Linear Matrix Inequality (LMI) framework. Initially, the ResNet architecture was reformulated as a cyclic tridiagonal LMI, and closed-form constraints on network parameters were derived to ensure $\mathcal{L}$-Lipschitz continuity; however, using a new $LDL^\top$ decomposition approach for certifying LMI feasibility, we extend the construction of $\mathcal{L}$-Lipchitz networks to any other nonlinear architecture. Our contributions include a provable parameterization methodology for constructing Lipschitz-constrained residual networks and other hierarchical architectures. Cholesky decomposition is also used for efficient parameterization. These findings enable robust network designs applicable to adversarial robustness, certified training, and control systems. The $LDL^\top$ formulation is shown to be a tight relaxation of the SDP-based network, maintaining full expressiveness and achieving 3\%-13\% accuracy gains over SLL Layers on 121 UCI data sets.

翻译：深度残差网络（ResNets）在计算机视觉任务中取得了显著成功，这归因于其能够通过深度架构保持梯度流动。同时，控制神经网络中的Lipschitz常数已成为增强对抗鲁棒性和网络可验证性的关键研究领域。本文提出了一种基于线性矩阵不等式（LMI）框架的$\mathcal{L}$-Lipschitz深度残差网络通用设计的严格方法。首先，将ResNet架构重新表述为循环三对角LMI，并推导出网络参数的闭式约束以确保$\mathcal{L}$-Lipschitz连续性；然而，通过采用新的$LDL^\top$分解方法来验证LMI可行性，我们将$\mathcal{L}$-Lipschitz网络的构建扩展到任何其他非线性架构。我们的贡献包括一种可证明的参数化方法，用于构建Lipschitz约束的残差网络及其他分层架构。同时利用Cholesky分解实现高效参数化。这些发现支持适用于对抗鲁棒性、可验证训练和控制系统的鲁棒网络设计。$LDL^\top$公式被证明是基于半定规划（SDP）网络的紧松弛，保持了完全表达能力，并在121个UCI数据集上相比SLL层实现了3%-13%的准确率提升。