---
title: Periodic Stochastic Game-Theoretic Riccati Equations
url: https://www.emergentmind.com/topics/periodic-time-varying-stochastic-game-theoretic-riccati-differential-equations
type: topic
---

# Periodic Stochastic Game-Theoretic Riccati Equations

Periodic time-varying stochastic game-theoretic Riccati differential equations (SGTRDEs) constitute a class of matrix-valued nonlinear differential equations that arise in the optimal control and saddle-point analysis of zero-sum linear–quadratic stochastic differential games (LQ-SDGs) with periodic, time-dependent system data. These equations encode the value function and the optimal policies of two adversarial controllers interacting over a stochastic Itô system with both drift and diffusion coefficients being $T$-periodic functions of time. The stabilizing periodic solutions to SGTRDEs determine global saddle-point optimality criteria and feedback synthesis for high-dimensional, time-varying stochastic systems.

## 1. Mathematical Formulation and Theoretical Foundations

Consider a filtered probability space supporting an $r$-dimensional Brownian motion $W(t)$, and the state evolution governed by the controlled Itô dynamics:
\[
dx(t) =
\left(
  A_0(t)x(t) + B_{01}(t)u_1(t) + B_{02}(t)u_2(t)
\right)dt 
+ \sum_{k=1}^r \left(
  A_k(t)x(t) + B_{k1}(t)u_1(t) + B_{k2}(t)u_2(t)
\right)dw_k(t),
\quad x(0) = x_0 \in \mathbb{R}^n,
\]
where $u_1 \in \mathbb{R}^{m_1}$ (maximizer) and $u_2 \in \mathbb{R}^{m_2}$ (minimizer) are control strategies. The corresponding cost functional of the zero-sum game is
\[
J(x_0; u_1, u_2) = \mathbb{E}\int_0^\infty 
\begin{pmatrix}
x(t)\\
u_1(t)\\
u_2(t)
\end{pmatrix}^\top
\begin{pmatrix}
M(t) & L_1(t) & L_2(t)\\
L_1(t)^\top & R_{11}(t) & R_{12}(t)\\
L_2(t)^\top & R_{12}(t)^\top & R_{22}(t)
\end{pmatrix}
\begin{pmatrix}
x(t)\\
u_1(t)\\
u_2(t)
\end{pmatrix}
dt,
\]
where all data matrices are $T$-periodic and continuous. Saddle-point (value) solutions are determined by a symmetric, matrix-valued function $X(t)$ which satisfies a coupled, nonlinear Riccati-type matrix differential equation (the SGTRDE) with the periodic boundary condition $X(t + T) = X(t)$. The sign-indefinite control weighting structure is encoded by
\[
\operatorname{sgn}\left[R(t) + \sum_k B_k(t) X(t) B_k(t)\right] = \operatorname{diag}(-I_{m_1}, I_{m_2}),
\]
reflecting the maximizing ($u_1$) and minimizing ($u_2$) roles.

The full SGTRDE is
\[
\begin{aligned}
\dot X
&+ A_0 X + X A_0^\top + \sum_k A_k X A_k^\top + M \\
&- \left(X B_0 + \sum_k A_k X B_k + L\right)\left(R + \sum_k B_k X B_k\right)^{-1}\left( B_0^\top X + \sum_k B_k^\top X A_k + L^\top \right) = 0,
\end{aligned}
\]
where $B_0 = [B_{01},\; B_{02}]$ and $B_k = [B_{k1},\; B_{k2}]$. The domain is restricted by the quadratic sign-definiteness: $R_{22}(t) + \cdots > 0$, $R_{11}(t) + \cdots < 0$.

Theoretical soundness relies on stochastic stabilizability and detectability notions: an Itô system is stochastically stabilizable if there exists a $T$-periodic feedback making all closed-loop modes mean-square exponentially stable, and stochastically detectable under similar output criteria. Existence and uniqueness of periodic stabilizing solutions are contingent on these properties and the definiteness of $R_{22}(t)$.

## 2. Dual-Layer Iterative Solution Framework

Directly solving the periodic SGTRDE is complicated by its fully coupled, sign-indefinite, nonlinear structure. The introduced solution methodology reformulates the problem as a sequence of bilevel (dual-layer) interconnected subproblems expressed as interlaced iterates of matrix-valued functions: an "outer" sequence $X^{(h)}(t)$ and an "inner" sequence $Z^{(h)}(t)$, with $h=0,1,2,\dots$.

- **Initialization**: $X^{(0)}(t) \equiv 0$, $Z^{(0)}(t)$ is the unique $T$-periodic stabilizing solution of a definite-sign inner Riccati equation based on the open-loop case.
- **Outer Update**: $X^{(h)}(t) = X^{(h-1)}(t) + Z^{(h-1)}(t)$.
- **Inner Update**: $Z^{(h)}(t)$ is obtained as the unique $T$-periodic stabilizing solution of a Riccati equation with updated coefficients $A_k^{(h)}(t) = A_k(t) + B_k(t)F(t, X^{(h)}(t))$ and an additional correction term $V^{(h)}(t)$.

The correction term $V^{(h)}(t)$ and feedback mapping are given by
\[
F(t, X) = -\left[R + \sum_k B_k X B_k\right]^{-1}\left(B_0^\top X + \sum_k B_k^\top X A_k + L^\top\right),
\]
\[
V^{(h)}(t) = \left(I_{m_1}, -R_{12}R_{22}^{-1}\right)\left[B_{01} Z^{(h-1)} + \sum_k B_{k1}Z^{(h-1)}(A_k + B_k F)\right].
\]

This iterative mechanism produces a monotone, non-decreasing sequence of outer approximants, with each inner solution computed for a definite-sign Riccati problem conditioned on the current guess.

## 3. Algorithmic Implementation and Workflow

The proposed algorithm to compute the stabilizing periodic solution $X^*(t)$ is as follows:

1. **Initialization**: Set $X^{(0)}(t) \leftarrow 0$.
2. **Compute Initial Inner Solution**: Solve the inner Riccati DE with definite quadratic sign to obtain $Z^{(0)}(t)$.
3. **Iterative Update**: For $h = 1, 2, \dots$:
    - $X^{(h)}(t) = X^{(h-1)}(t) + Z^{(h-1)}(t)$,
    - Compute $F^{(h)}(t) = F(t, X^{(h)}(t))$,
    - Compute $V^{(h)}(t)$,
    - Solve inner Riccati (9) for new $Z^{(h)}(t)$.
4. **Stopping Criterion**: Terminate when $\sup_{t\in[0,T]}\|Z^{(h)}(t)\| < \varepsilon$ for prescribed tolerance $\varepsilon$.
5. **Output**: $X^*(t) \approx X^{(h)}(t)$.

This approach ensures the accumulation of monotone corrections and, under appropriate stabilizability and detectability conditions, iteratively converges to the global stabilizing periodic solution.

### Summary Table: Iterative Algorithm Structure

| Step            | Description                           | Output                                  |
|-----------------|---------------------------------------|------------------------------------------|
| Initialization  | Set $X^{(0)}(t) \leftarrow 0$         | Initial outer approximation              |
| Inner Solution  | Solve definite-sign Riccati for $Z^{(h)}(t)$ | Correction term for $X^{(h)}$        |
| Outer Update    | $X^{(h)}(t) = X^{(h-1)}(t) + Z^{(h-1)}(t)$ | Updated solution candidate           |
| Stopping Rule   | Stop if $\|Z^{(h)}(t)\| < \varepsilon$ | Final stabilizing solution              |

## 4. Analysis of Convergence and Theoretical Guarantees

The convergence analysis is grounded in domain invariance, monotonicity, and boundedness of correction sequences. Key statements:

- **Domain Invariance**: If $R_{22}(t) > 0$, the iterates and auxiliary solutions remain within the domain of well-posedness for the generator $\mathcal{G}$.
- **Auxiliary Existence**: If the linear part is stochastically stabilizable/detectable and $M - L R^{-1} L^\top \succ 0$, each inner Riccati subproblem admits a unique $T$-periodic stabilizing solution.
- **Monotonicity**: If the outer iterate yields a stable closed-loop, then the new inner solution $\widetilde{Y}_{K,W}$ dominates the previous iterate; thus, $X^{(0)} \le X^{(1)} \le \cdots \le \widetilde{Y}_{K,W}$.
- **Global Convergence**: Under the above structural assumptions, the sequences are globally convergent:
    - $\lim_{h \to \infty} Z^{(h)} = 0$,
    - The limit $X^*(t) = \lim_{h \to \infty} X^{(h)}(t)$ is the unique $T$-periodic stabilizing solution of the original SGTRDE.

The proof combines contraction arguments for auxiliary Riccati flows and monotone operator theory in the space of periodic symmetric matrix-valued functions.

## 5. Empirical Performance and Numerical Evaluation

Large-scale Monte Carlo experiments validate the iteration framework. For system orders $n = 1, \dots, 20$, 1,000 random trials per $n$ (totaling 20,000) are conducted. Data is generated with:

- $A_k \sim \mathcal{N}(0,1)$,
- $B_{0i} = 3I \pm 0.5H_i$,
- $B_{ki} \sim U[0,0.01]$,
- $R_{11} = -4I - U_{11} U_{11}^\top$,
- $R_{22} = 5I + U_{22} U_{22}^\top$,
- $L, M$ chosen to ensure $M - L R^{-1} L^\top \succ 0$,
- Fixed period $T=1$, with MATLAB default random seed.

Key observations:
- The required number of outer iterations for convergence (tolerance $1\mathrm{e}{-8}$) is typically $8$–$13$, exceeding $13$ in only $2$ out of $20,000$ trials.
- Inner iteration counts per outer step increase as the algorithm approaches stationarity.
- Low-dimensional systems exhibit higher variability in inner iteration counts, whereas higher-dimensional cases stabilize rapidly.

Qualitative histograms demonstrate consistent convergence behavior across dimensions and random instances, supporting the practical robustness of the dual-layer iteration scheme.

## 6. Applications and Broader Implications

Solutions to periodic SGTRDEs underpin feedback synthesis for zero-sum stochastic games with periodic coefficients, such as those encountered in financial engineering, signal processing, and systems with seasonal or cyclic behaviors. The presented algorithm provides a unified, numerically stable framework applicable to a broad class of such problems without restrictive simplifications or ad hoc regularization. The explicit dual-layer structure allows decomposition into tractable, definite-sign Riccati subproblems at each step, facilitating both theoretical analysis and scalable implementation.

A plausible implication is that the generality of this framework enables systematic studies and controller synthesis for new subclasses of time-periodic stochastic control problems, potentially extending to more complex multi-agent or non-zero-sum games, given further investigation of analogous structural properties.

Source: https://www.emergentmind.com/topics/periodic-time-varying-stochastic-game-theoretic-riccati-differential-equations