StableUN: Robust Stability in Learning Systems

• StableUN is a framework that enforces explicit stability constraints in various learning systems including deep unrolled networks, reinforcement learning, data unfolding, and LLM unlearning.
• The methodology uses stochastic descent constraints, dynamic regularization, and Lyapunov cost shaping to ensure convergence and resistance to noise and perturbations.
• Key applications span LISTA for sparse coding, image inpainting with GLOW-Prox, stabilized reinforcement learning, and robust unlearning against adversarial attacks.

StableUN refers to multiple distinct frameworks centered on the principle of stability in learning systems, robust network optimization, dynamical data unfolding, and unlearning in large-scale models. The term designates different mechanisms across deep unrolled architectures, iterative data reconstruction, stable reinforcement learning in unbounded domains, and feedback-guided optimization for robust unlearning. Underlying these usages is the formalization and enforcement of stability—either as descent, resistance to perturbation, resilience to data/model misspecification, or immunity to adversarial reversal—through explicit constraints or algorithmic structures. The following sections systematically review and compare these approaches.

1. Stochastic Descent Constraints in Deep Unrolled Networks

"StableUN" in the context of deep unrolled networks denotes architectures trained with explicit stochastic descent constraints at each layer, as described by robust stochastically-descending unrolled networks (Hadou et al., 2023). In this setting, a family of smooth objectives $f(y; x)$ is optimized via a parameterized, finite-depth network $\Phi(x; W)$, where intermediate outputs $y_l$ emulate steps of an underlying iterative algorithm. The learning problem is formulated as:

$$\min_W\, \mathbb{E}\left[\ell(\Phi(x; W), y^\ast)\right] \quad \text{s.t.} \quad \mathbb{E}[C(y_l, y_{l-1})] \leq 0,~ l=1, \ldots, L,$$

where $C$ is a per-layer stochastic descent constraint. Two equivalent forms enforce either:

• A gradient-norm descent: $$\mathbb{E}\left[\|\nabla f(y_l; x)\|_2 - (1-\epsilon)\|\nabla f(y_{l-1}; x)\|_2\right] \leq 0$$,
• Or proximity to optimum: $$\mathbb{E}\left[\|y_l-y^\ast\|_2 - (1-\epsilon)\|y_{l-1}-y^\ast\|_2\right] \leq 0$$.

Gaussian noise is injected at each layer to ensure statistical soundness of the constraints. This construction ensures that each unrolled layer, in expectation, advances toward the optimum.

2. Theoretical Guarantees: Convergence and Robustness

StableUN provides explicit convergence guarantees under the stochastic descent constraints, a feature not present in standard deep unrolling. The central result establishes that, under mild conditions (smoothness of $f$, functional expressivity, strict feasibility of constraints), the sequence of outputs $\{y_l\}$ generated by the trained network satisfies:

$$\lim_{l\to\infty} \mathbb{E}\left[\min_{k\leq l} \|\nabla f(y_k;x)\|\right] \leq \frac{1}{\epsilon}[\zeta(N,\delta)+\delta M/(1-\delta)] \quad \text{a.s.}$$

where $\Phi(x; W)$0 as the sample size increases and $\Phi(x; W)$1 is the Lipschitz constant. The rate is geometric up to a bias term determined by constraint slackness and noise. This convergence is robust to layerwise perturbations: at inference, additive noise or adversarial deviations at any layer cause only bounded degradation, as each subsequent layer is constrained to recover stochastically. In comparison, standard (unconstrained) unrolling cannot provide such recoverability, since intermediate progress is unregulated (Hadou et al., 2023).

3. Algorithmic Realizations and Applications

StableUN's descent-constrained learning is instantiated in two canonical unrolled architectures:

• LISTA for Sparse Coding: The architecture emulates the LASSO objective and iteratively applies soft-thresholding with learned parameters. The distance-to-optimum constraint is imposed at every layer, encouraging steady proximity to ground truth. Noise is added to hidden activations to decorrelate constraints.
• GLOW-Prox for Image Inpainting: The network solves a latentspace MAP estimation, alternating between proximal and generative steps. Both gradient-norm constraints on the data-fidelity and latent prior terms are enforced. Training follows a primal–dual methodology involving augmented Lagrangian optimization over both weights and dual multipliers.

For both settings, constrained training requires backpropagating through the constraint terms and updating dual variables in a projected gradient ascent manner. At inference, the noise and dual variables are omitted (Hadou et al., 2023).

4. Dynamically Stabilized Data Unfolding

In high-energy physics, "StableUN" labels a dynamically stabilized iterative data unfolding method employing regularization functions to:

• Suppress transfer of statistical fluctuations beyond their original bin and
• Enable unbiased reconstruction of new structures absent from the Monte Carlo simulation.

The algorithm iteratively normalizes the MC prediction, unfolds observed spectra using a smooth threshold function $\Phi(x; W)$2 quantifying statistical significance, subtracts detected background fluctuations, and improves the response matrix. By tuning regularization parameters, the user controls the induced bin-to-bin correlations, achieving stability against both overfitting to statistical noise and the propagation of MC/model bias (0907.3791).

Empirically, this yields robust, low-bias unfolded spectra, with convergence and error/covariance estimable via analytic propagation or toy MC studies.

5. Stability in Reinforcement Learning on Unbounded State Spaces

In reinforcement learning, "StableUN" refers to an approach (originally termed STOP) for stabilizing online policy learning in unbounded state spaces (Pavse et al., 2023). The framework integrates:

• Lyapunov-based cost shaping: Augmenting the reward with the change in a Lyapunov function $\Phi(x; W)$3, e.g., $\Phi(x; W)$4, to penalize destabilizing actions severely and sharpen credit assignment.
• Coordinate-wise state compression: Applying order-preserving, sublinear transformations like $\Phi(x; W)$5 to prevent neural networks from extrapolating in rarely-visited regions.

This yields agents that, despite being trained without explicit model knowledge, exhibit empirically stabilized trajectories and bounded costs in queuing and control problems. Theoretical analysis shows that cost shaping preserves optimal policies under mild growth conditions.

6. Robust Neighborhood-Aware Unlearning in LLMs

In large-scale model unlearning, StableUN denotes a bi-level feedback-guided optimization framework that addresses failures of conventional forgetting-by-gradient methods, which tend to converge to sharp minima—regions in parameter space where small perturbations can easily restore deleted knowledge (relearning attacks) (Wu et al., 24 Sep 2025). StableUN achieves robust unlearning by:

• Neighborhood-aware minimax optimization: The objective

$\Phi(x; W)$6

explicitly seeks parameter regions where forgetting is robust to small adversarial changes.

• Forgetting feedback: At each optimization step, adversarial or stochastic perturbations are generated and the worst-case forgetting loss is used to drive the model away from sharp basins.
• Remembering feedback with gradient harmonization: Regular utility-preserving feedback is computed on retained or public data. When the forgetting and remembering gradients are antagonistic, the forgetting gradient is projected onto the orthogonal complement of the utility gradient, yielding an update that preserves both objectives.

Empirical results demonstrate that StableUN significantly improves resistance to relearning and jailbreaking attacks, reducing post-attack memory retention metrics by more than 50% compared to vanilla gradient- or KL-based methods while closely matching their utility performance.

7. Comparative Summary and Theoretical Connections

A unifying theme across these usages of StableUN is the formalization and enforcement of per-iteration or per-layer stability, either through Lagrangian-constrained optimization (deep unrolling), statistical significance functions (data unfolding), cost shaping and state compression (RL), or adversarially-aware bi-level objectives with feedback harmonization (LLM unlearning).

Context	Stability Mechanism	Principal Guarantee
Unrolled networks (Hadou et al., 2023)	Stochastic descent constraint per layer	Convergence + robustness to noise
Data unfolding (0907.3791)	Dynamic regularization in iteration	Bias control + statistical stabilization
RL in unbounded spaces (Pavse et al., 2023)	Lyapunov cost shaping, feature compression	Empirical stabilization + bounded cost
LLM unlearning (Wu et al., 24 Sep 2025)	Neighborhood-aware, harmonized feedback	Robust forgetfulness, attack resistance

Stability, in these frameworks, is not merely an empirical property but a guarantee enforced by explicit mathematical constructs operating at every step or layer. This consistently yields resilience to noise, model misspecification, adversarial attacks, and trajectory perturbations. Theoretical analyses uniformly emphasize that mere terminal objective optimization is insufficient for robust performance; per-update constraints or compositional harmonization are required to ensure system-wide stability and generalization.