Adversarial Generalization of Unfolding Networks

• The paper derives provable adversarial generalization error bounds for unfolding networks using an ARC framework that incorporates attack strength within logarithmic scaling.
• It demonstrates that overparameterization via redundant sparsifying operators significantly enhances robustness against FGSM adversarial perturbations, as validated on datasets like CIFAR10 and SVHN.
• This work explores the trade-off between network depth, redundancy, attack intensity, and sample size, offering actionable insights for designing robust model-based architectures.

Adversarial generalization of unfolding networks refers to the quantitative and structural understanding of how unfolded or model-based neural networks—derived by unrolling iterative algorithms—perform under adversarial attacks, particularly in critical inverse problems such as compressed sensing. Unfolding networks combine domain-based priors (e.g., sparsity, analysis operators) with parameterized learning, achieving high interpretability and accuracy for recovering signals from incomplete or noisy data. In contrast to traditional deep networks, a theoretical account for their behavior under adversarial perturbations, especially with provable error bounds and practical robustness mechanisms, has only recently begun to emerge.

1. Theoretical Foundations: Adversarial Rademacher Complexity and Generalization Bounds

The core theoretical framework rests on the introduction of adversarial Rademacher complexity (ARC) for classes of unfolding networks, especially those employing overparameterized, redundant sparsifying operators. ARC extends the classical notion of Rademacher complexity—used to control generalization gaps in the i.i.d. regime—by considering the supremum of signed empirical averages over an adversarial hypothesis class, wherein each element is a perturbed network output: $$\mathcal{R}_s(\widetilde{\mathcal{H}}^L) = \mathbb{E}_{\varepsilon}\sup_{\widetilde{h}\in\widetilde{\mathcal{H}}^L} \frac{1}{s} \sum_{i=1}^s \varepsilon_i \widetilde{h}(y_i)$$ with $\widetilde{\mathcal{H}}^L$ parameterizing the class under adversarial perturbations generated via, for example, the fast gradient sign method (FGSM).

A principal result is the derivation of adversarial generalization error bounds of the form: $$\widetilde{GE}(\widetilde{h}) \leq \mathcal{O}\left( \sqrt{\frac{NL\log(\epsilon)}{s}} \right)$$ Here, $N$ is the overcompleteness of the analysis operator, $L$ is the number of unfolded layers, $\epsilon$ is the adversarial attack level (radius in $\ell_2$ norm), and $s$ is the sample size. The bound tightly quantifies how generalization error under adversarial input grows with both architecture capacity and attack magnitude, and crucially, contains the attack strength $\epsilon$ inside a logarithmic term, reflecting that increased attack budget leads to more complex adversarial hypothesis classes and hence looser generalization bounds (Kouni, 18 Sep 2025).

This is achieved by:

• Proving Lipschitz continuity of the perturbed decoder $h_W^L(y+\delta)$ with respect to the learnable parameters (e.g., overcomplete analysis operator $W$).
• Leveraging Dudley’s entropy-integral and covering number estimates of the hypothesis class under adversarial perturbation.
• Explicitly relating the covering numbers to the Lipschitz constant of the decoder, which itself depends at most exponentially on network depth $L$ and linearly on $\epsilon$ inside the logarithmic covering number (Kouni, 18 Sep 2025).

2. Adversarial Attack Modeling and Hypothesis Class Construction

The adversarial attacks considered are constrained in the $\ell_2$ norm and constructed using FGSM: $$\delta_{W}^{\mathrm{FGSM}} = \epsilon \cdot \frac{\nabla_y \| h_W(y) - x \|_2^2}{\| \nabla_y \| h_W(y) - x \|_2^2 \|_2}$$ where $\epsilon$ specifies the maximum allowable perturbation magnitude. This attack is generated in a white-box setting: the adversary has access to all network parameters. The hypothesis class $\widetilde{\mathcal{H}}^L$ thus consists of all decoders $h_W^L$ evaluated at adversarially perturbed inputs $y + \delta_W^{\mathrm{FGSM}}$ (Kouni, 18 Sep 2025).

As $\epsilon$ increases, the network’s effective Lipschitz constant $Lip_h^{(L,\epsilon)}$ increases, directly impacting the entropy (covering number) of the adversarial class and thus generalization error. This interplay is foundational to both the theory and design of adversarially robust unfolding architectures.

3. Experimental Validation: Scaling Laws and Robustness via Overparameterization

Empirical studies confirm that adversarial generalization matches the predicted scaling of the error bound. Experiments on real-world datasets (e.g., CIFAR10, SVHN) show:

• The clean and adversarial test mean squared error (MSE) both increase with attack magnitude $\epsilon$ but with controlled degradation, indicative of non-catastrophic error escalation.
• The adversarial empirical generalization error (test-train gap under adversarial perturbation) scales approximately as $\sqrt{L\log(\epsilon)}$, confirming the derived upper bounds in practice.
• Increasing overparameterization—taking a larger $N$ for the redundant analysis operator—decreases adversarial test error and narrows the generalization gap, indicating that judicious overparameterization enhances robustness to input attacks.
• Comparisons with traditional (less overparameterized) architectures, such as ISTA-net (which uses an orthogonal sparsifier), demonstrate marked improvements in robustness and clean/adversarial MSE by the overparameterized model-based (e.g., ADMM-DAD) unfolding networks (Kouni, 18 Sep 2025).

4. Architectural Implications: Overparameterization and Robustness

A principal structural insight is that the overparameterization induced by redundant sparsifiers (e.g., $N > n$ where $N$ is number of rows in $W$ and $n$ signal dimension) can be directly exploited for adversarial robustness. The bound

$$\widetilde{GE}(\widetilde{h}) = \mathcal{O}\left( \sqrt{\frac{NL\log(\epsilon)}{s}} \right)$$

suggests that increasing $N$ and $L$ can lead to greater network capacity and accuracy but at the expense of potentially higher adversarial error unless sample size $s$ is also scaled. However, moderate overparameterization (with appropriate regularization) can yield both better clean and adversarial generalization—reconciling the classic capacity–robustness dilemma.

This effect aligns with results on generalization in unfolded and compound Gaussian networks (Lyons et al., 20 Feb 2024), where covering number analysis via Rademacher complexity demonstrates robustness gains when the effective hypothesis class is constrained by priors or overparameterized operators.

5. Relationship to Broader Robustness and Generalization Theory

This work builds on the emerging understanding of adversarial generalization in deep models (Kouni et al., 2022), extending classical generalization error bounds from the i.i.d. setting to adversarially perturbed input distributions. Previous results for unfolding networks established generalization error bounds in the clean setting—typically scaling as $\sqrt{NL/s}$ or $\mathcal{O}(n\sqrt{\ln(n)})$ for compound Gaussian unrolled networks (Lyons et al., 20 Feb 2024). By extending these to adversarial regimes, it is now possible to certify, for the first time, that unfolding architectures can reliably function under adversarial input perturbations, provided their structural and training parameters are appropriately chosen (Kouni, 18 Sep 2025).

Connections with recent theory—such as the role of generalization in transferability of adversarial examples (Wang et al., 2022), the effect of overfitting in robust feature learning (Lee et al., 2020), and regularization approaches for model-based networks (Kouni et al., 2023, Kouni et al., 2022)—are now mathematically formalized within this more general adversarial complexity framework.

6. Practical Implications and Future Directions

The derived theory and empirical validation provide architectural and operational guidance:

• When deploying unfolding networks in adversarially exposed applications (such as medical imaging or cryptography), overparameterization should be leveraged to enhance robustness, balanced by sample size and regularization to avoid excessive complexity.
• The error bound quantifies the tradeoff between depth, redundancy, attack intensity, and data efficiency: deeper, wider nets can be robust if sufficient training data is available and regularization is enforced.
• The ARC-based framework opens avenues for extending results to broader attack models (e.g., PGD, $\ell_\infty$), for analyzing dynamic or adaptive unrolling strategies, and for exploring tighter, possibly instance-dependent, generalization bounds.
• The insight that moderate overparameterization confers robustness—previously a heuristic principle—is now substantiated by precise complexity–generalization theory, helping to close the gap between robust learning in black-box architectures and interpretable, model-based unfolding networks.

Future research may focus on tightening ARC-derived bounds, exploring other forms of adversarial training or input/output perturbation schemes, and further investigating the interplay between network structure (frames, analysis vs synthesis models), sample complexity, and adversarial risk.

7. Summary Table: Key Quantities in Adversarial Generalization of Unfolding Networks

Symbol/Term	Description	Scaling/Role
$N$	Overcompleteness of analysis operator $W$	Higher $N$ promotes robustness
$L$	Number of unfolded layers	Increases capacity, error scales as $\sqrt{L}$
$\epsilon$	Attack level (FGSM $\ell_2$ norm)	Error scales as $\sqrt{\log(\epsilon)}$
$s$	Sample size	Error decreases as $1/\sqrt{s}$
$Lip_h^{(L,\epsilon)}$	Lipschitz constant of perturbed decoder	Depends exponentially on $L$ and linearly on $\epsilon$ inside log
$\widetilde{GE}(\widetilde{h})$	Adversarial generalization error bound	$\mathcal{O}(\sqrt{N L \log(\epsilon)/s})$

The above encapsulates the main quantities controlled or analyzed within the adversarial generalization framework for model-based (unfolding) networks, as established in (Kouni, 18 Sep 2025). Theoretical and empirical findings converge, providing robust design principles and paving the way for further advances in provably resilient architectures for inverse problems.