---
title: 'RNVI: Regularized Normalized Value Iteration'
url: https://www.emergentmind.com/topics/regularized-normalized-value-iteration-rnvi
type: topic
---

# RNVI: Regularized Normalized Value Iteration

Regularized Normalized Value Iteration (RNVI) is a fixed-point iterative algorithm developed to compute both near-optimal state-feedback controllers and rigorous two-sided bounds for the optimal mean-square stabilizing rate, $\rho^*$, in stochastic discrete-time linear systems with multiplicative noise. The method reformulates the classic stabilization objective as a nonlinear matrix eigenvalue problem augmented by regularization and normalization, ensuring the existence and strict positivity of Lyapunov-type solutions even in high-dimensional or degenerate settings. RNVI synthesizes feedback gains and certified performance guarantees while maintaining computational tractability for moderately large-scale systems [2512.06349].

## 1. Nonlinear Eigenvalue Formulation

The stochastic discrete linear system under consideration is governed by state transition matrices $(A, \bar A)$ and control matrices $(B, \bar B)$, alongside a noise process $\omega \sim \mathcal N(0,\sigma^2)$. Restricting to time-invariant state-feedback laws $u_k = -K x_k$, any quadratic Lyapunov-type certificate $h(x) = x^\top P x$, $P \in \mathbb S^n_{++}$, must satisfy the Bellman-type equality:
\[
e^{\lambda} h(x) = \inf_{v \in \mathbb R^m} \mathbb E \left[h\big((A+\bar A \omega)x + (B+\bar B \omega)v\big) \,\big|\, x \right].
\]
Optimization over $v$ yields the so-called stochastic Riccati blocks:
\[
R(P) = B^\top P B + \sigma^2 \bar B^\top P \bar B,\quad S(P) = A^\top P B + \sigma^2 \bar A^\top P \bar B,
\]
and the nonlinear operator:
\[
\Phi(P) = A^\top P A + \sigma^2 \bar A^\top P \bar A - S(P) R(P)^{-1} S(P)^\top.
\]
The stabilization rate problem is then equivalent to the nonlinear matrix eigenvalue problem (EV):
\[
\Phi(P) = \gamma P, \quad \gamma = e^\lambda, \quad P \succ 0.
\]

## 2. Regularization and Normalization Approach

The principal challenge is that (EV) may only admit positive semi-definite solutions on the boundary of $\mathbb S^{n}_+$, rendering them inadequate for certifying exponential stabilization or synthesizing controllers. To guarantee strictly positive-definite matrices and keep iterates interior to the cone, the operator $\Phi$ is perturbed and normalized using a regularization parameter $\tau \in (0,1)$:
\[
Y_\tau(P) = (1 - \tau) \Phi(P) + \frac{\tau}{n} I,\qquad
\widehat\Phi_\tau(P) = \frac{Y_\tau(P)}{\mathrm{Tr}(Y_\tau(P))}.
\]
This regularization mixes the image of $\Phi(P)$ with the full-rank matrix $\frac{1}{n}I$ and ensures that the mapping $\widehat\Phi_\tau$ preserves the compact, convex trace slice $\{P \succeq 0 : \mathrm{Tr} P = 1\}$.

## 3. RNVI Algorithmic Procedure

RNVI is specified as a fixed-point map on positive-definite matrices with unit trace:
\[
P_{k+1} = \widehat\Phi_\tau(P_k),\quad P_0 \in \mathbb S^n_{++},\; \mathrm{Tr}(P_0) = 1.
\]
The update steps are:
```matlab
Input: (A, Ā, B, Ḃ), noise variance σ², τ ∈ (0,1), initial P₀ ≻ 0, Tr(P₀) = 1.
For k = 0,1,2,...:
   1. Rₖ = BᵀPₖB + σ²·ḂᵀPₖḂ;  Sₖ = AᵀPₖB + σ²·ĀᵀPₖḂ.
   2. Φₖ = AᵀPₖA + σ²·ĀᵀPₖĀ – Sₖ·Rₖ⁻¹·Sₖᵀ.
   3. Yₖ = (1–τ)·Φₖ + (τ/n)·I.
   4. Trₖ = trace(Yₖ).
   5. Pₖ₊₁ = Yₖ / Trₖ.
   6. If ∥Pₖ₊₁ – Pₖ∥ ≤ ε, stop.
End For
Output: P^(τ) ≈ fixed point of widehatΦ₍τ₎.
```
Each update guarantees $P_k \in \mathbb S^{n}_{++}$ with $\mathrm{Tr} P_k = 1$.

## 4. Existence and Convergence Properties

- **Existence**: Brouwer’s theorem ensures that, for each $\tau \in (0,1)$, the map $\widehat\Phi_\tau$ admits a fixed point $P^{(\tau)} \succ 0$ with $\mathrm{Tr} P^{(\tau)} = 1$. All eigenvalues are uniformly bounded away from zero: $P^{(\tau)} \succeq \delta_\tau I$ for some $\delta_\tau > 0$.
- **Contraction and Rate**: Whenever the local Lipschitz constant $\Lambda(\tau)$ satisfies $\Lambda(\tau) < 1$, the map is a contraction, guaranteeing global linear convergence:
  \[
  \|P_k - P^{(\tau)}\| \leq \Lambda(\tau)^k \|P_0 - P^{(\tau)}\|.
  \]
- **Continuation in $\tau$**: If $\Lambda(\tau)\geq 1$, convergence is attained by parameter continuation: starting from a “safe” large $\tau_0$, RNVI is run to convergence, $\tau$ is reduced (e.g., halved) at each step, and each fixed point is used to initialize the next run. Local contraction arguments ensure success of this procedure.

## 5. Feedback Controller Synthesis and Certified Bounds

Upon obtaining $P^{(\tau)}$, the corresponding state-feedback gain is
\[
K^{(\tau)} = R(P^{(\tau)})^{-1} S(P^{(\tau)})^\top.
\]
The algorithm additionally produces certified lower and upper bounds for the optimal stabilization performance. Define
\[
L(P) := \lambda_{\min}(P^{-1/2}\Phi(P)P^{-1/2}),\qquad
U(P) := \lambda_{\max}(P^{-1/2}\Phi(P)P^{-1/2}).
\]
Let $J^*$ be the infimum of the rate objective, $J^* = 2\log\rho^*$. For any feedback policy $u$, $J(x_0,u) \geq \log L(P)$, and under $u^{(\tau)}: u_k = - K^{(\tau)} x_k$, $J(x_0,u^{(\tau)}) \leq \log\left(\frac{\gamma^{(\tau)}}{1 - \tau}\right)$ where $\gamma^{(\tau)} = \mathrm{Tr}((1-\tau)\Phi(P^{(\tau)}) + \frac{\tau}{n}I)$. Thus, the true rate $\rho^*$ is bounded by:
\[
\sqrt{L(P^{(\tau)})} \leq \rho^* \leq \sqrt{\frac{\gamma^{(\tau)}}{1 - \tau}}.
\]
A $\tau$-sweep (Algorithm 1) identifies the tightest certified bounds by collecting results across a regularization path.

## 6. Computational Complexity and Practical Aspects

Each RNVI iterate comprises:
- Two quadratic-form computations ($A^\top P A$, $\bar A^\top P \bar A$)
- Solving one $m \times m$ linear system $R(P)X = S(P)^\top$
- Computing a matrix trace and an $m$-dimensional matrix inversion

The per-iteration computational cost is $O(n^3 + m^3)$. Under contraction ($\Lambda(\tau) < 1$), achieving accuracy $\varepsilon$ requires $O(\ln(1/\varepsilon)/\ln(1/\Lambda(\tau)))$ steps. The continuation in $\tau$ typically involves $M = 10$–$50$ steps, with each local RNVI converging geometrically. The overall numerical complexity is $O(M n^3 \ln(1/\varepsilon))$, making RNVI feasible for state dimensions up to several hundred.

## 7. Significance and Scope

RNVI resolves the fundamental difficulty of well-posed control synthesis for stochastic systems with multiplicative noise, converting the intractable nonlinear spectral condition into an interior fixed-point iteration that is both computationally viable and rigorously certifiable. It guarantees strictly positive-definite certificates, supports systematic controller extraction, and yields verifiable performance bounds for the optimal mean-square stabilization rate. The regularization and normalization paradigm ensures robustness to ill-conditioning, and the algorithmic structure supports practical deployment in moderate-dimensional linear stochastic control scenarios [2512.06349].

Source: https://www.emergentmind.com/topics/regularized-normalized-value-iteration-rnvi