Periodic Stochastic Game-Theoretic Riccati Equations

• Periodic SGTRDEs characterize the value function and optimal control policies in zero-sum LQ stochastic differential games with periodic system data.
• The dual-layer iterative method decomposes complex, sign-indefinite Riccati equations into tractable subproblems, ensuring convergence under stochastic stabilizability and detectability.
• Empirical evaluations demonstrate robust convergence with low iteration counts, validating its use in applications like financial engineering and cyclic control systems.

Periodic time-varying stochastic game-theoretic Riccati differential equations (SGTRDEs) constitute a class of matrix-valued nonlinear differential equations that arise in the optimal control and saddle-point analysis of zero-sum linear–quadratic stochastic differential games (LQ-SDGs) with periodic, time-dependent system data. These equations encode the value function and the optimal policies of two adversarial controllers interacting over a stochastic Itô system with both drift and diffusion coefficients being $T$-periodic functions of time. The stabilizing periodic solutions to SGTRDEs determine global saddle-point optimality criteria and feedback synthesis for high-dimensional, time-varying stochastic systems.

1. Mathematical Formulation and Theoretical Foundations

Consider a filtered probability space supporting an $r$-dimensional Brownian motion $W(t)$, and the state evolution governed by the controlled Itô dynamics: $$dx(t) = \left( A_0(t)x(t) + B_{01}(t)u_1(t) + B_{02}(t)u_2(t) \right)dt + \sum_{k=1}^r \left( A_k(t)x(t) + B_{k1}(t)u_1(t) + B_{k2}(t)u_2(t) \right)dw_k(t), \quad x(0) = x_0 \in \mathbb{R}^n,$$ where $u_1 \in \mathbb{R}^{m_1}$ (maximizer) and $u_2 \in \mathbb{R}^{m_2}$ (minimizer) are control strategies. The corresponding cost functional of the zero-sum game is

$$J(x_0; u_1, u_2) = \mathbb{E}\int_0^\infty \begin{pmatrix} x(t)\ u_1(t)\ u_2(t) \end{pmatrix}^\top \begin{pmatrix} M(t) & L_1(t) & L_2(t)\ L_1(t)^\top & R_{11}(t) & R_{12}(t)\ L_2(t)^\top & R_{12}(t)^\top & R_{22}(t) \end{pmatrix} \begin{pmatrix} x(t)\ u_1(t)\ u_2(t) \end{pmatrix} dt,$$

where all data matrices are $T$-periodic and continuous. Saddle-point (value) solutions are determined by a symmetric, matrix-valued function $X(t)$ which satisfies a coupled, nonlinear Riccati-type matrix differential equation (the SGTRDE) with the periodic boundary condition $X(t + T) = X(t)$. The sign-indefinite control weighting structure is encoded by

$$\operatorname{sgn}\left[R(t) + \sum_k B_k(t) X(t) B_k(t)\right] = \operatorname{diag}(-I_{m_1}, I_{m_2}),$$

reflecting the maximizing ($u_1$) and minimizing ($u_2$) roles.

The full SGTRDE is

$$\begin{aligned} \dot X &+ A_0 X + X A_0^\top + \sum_k A_k X A_k^\top + M \ &- \left(X B_0 + \sum_k A_k X B_k + L\right)\left(R + \sum_k B_k X B_k\right)^{-1}\left( B_0^\top X + \sum_k B_k^\top X A_k + L^\top \right) = 0, \end{aligned}$$

where $B_0 = [B_{01},\; B_{02}]$ and $B_k = [B_{k1},\; B_{k2}]$. The domain is restricted by the quadratic sign-definiteness: $R_{22}(t) + \cdots > 0$, $R_{11}(t) + \cdots < 0$.

Theoretical soundness relies on stochastic stabilizability and detectability notions: an Itô system is stochastically stabilizable if there exists a $T$-periodic feedback making all closed-loop modes mean-square exponentially stable, and stochastically detectable under similar output criteria. Existence and uniqueness of periodic stabilizing solutions are contingent on these properties and the definiteness of $R_{22}(t)$.

2. Dual-Layer Iterative Solution Framework

Directly solving the periodic SGTRDE is complicated by its fully coupled, sign-indefinite, nonlinear structure. The introduced solution methodology reformulates the problem as a sequence of bilevel (dual-layer) interconnected subproblems expressed as interlaced iterates of matrix-valued functions: an "outer" sequence $X^{(h)}(t)$ and an "inner" sequence $Z^{(h)}(t)$, with $h=0,1,2,\dots$.

• Initialization: $X^{(0)}(t) \equiv 0$, $Z^{(0)}(t)$ is the unique $T$-periodic stabilizing solution of a definite-sign inner Riccati equation based on the open-loop case.
• Outer Update: $X^{(h)}(t) = X^{(h-1)}(t) + Z^{(h-1)}(t)$.
• Inner Update: $Z^{(h)}(t)$ is obtained as the unique $T$-periodic stabilizing solution of a Riccati equation with updated coefficients $A_k^{(h)}(t) = A_k(t) + B_k(t)F(t, X^{(h)}(t))$ and an additional correction term $V^{(h)}(t)$.

The correction term $V^{(h)}(t)$ and feedback mapping are given by

$$F(t, X) = -\left[R + \sum_k B_k X B_k\right]^{-1}\left(B_0^\top X + \sum_k B_k^\top X A_k + L^\top\right),$$

$$V^{(h)}(t) = \left(I_{m_1}, -R_{12}R_{22}^{-1}\right)\left[B_{01} Z^{(h-1)} + \sum_k B_{k1}Z^{(h-1)}(A_k + B_k F)\right].$$

This iterative mechanism produces a monotone, non-decreasing sequence of outer approximants, with each inner solution computed for a definite-sign Riccati problem conditioned on the current guess.

3. Algorithmic Implementation and Workflow

The proposed algorithm to compute the stabilizing periodic solution $X^*(t)$ is as follows:

1. Initialization: Set $X^{(0)}(t) \leftarrow 0$.
2. Compute Initial Inner Solution: Solve the inner Riccati DE with definite quadratic sign to obtain $Z^{(0)}(t)$.
3. Iterative Update: For $h = 1, 2, \dots$:
   • $X^{(h)}(t) = X^{(h-1)}(t) + Z^{(h-1)}(t)$,
   • Compute $F^{(h)}(t) = F(t, X^{(h)}(t))$,
   • Compute $V^{(h)}(t)$,
   • Solve inner Riccati (9) for new $Z^{(h)}(t)$.
4. Stopping Criterion: Terminate when $\sup_{t\in[0,T]}\|Z^{(h)}(t)\| < \varepsilon$ for prescribed tolerance $\varepsilon$.
5. Output: $X^*(t) \approx X^{(h)}(t)$.

This approach ensures the accumulation of monotone corrections and, under appropriate stabilizability and detectability conditions, iteratively converges to the global stabilizing periodic solution.

Summary Table: Iterative Algorithm Structure

Step	Description	Output
Initialization	Set $X^{(0)}(t) \leftarrow 0$	Initial outer approximation
Inner Solution	Solve definite-sign Riccati for $Z^{(h)}(t)$	Correction term for $X^{(h)}$
Outer Update	$X^{(h)}(t) = X^{(h-1)}(t) + Z^{(h-1)}(t)$	Updated solution candidate
Stopping Rule	Stop if $\|Z^{(h)}(t)\| < \varepsilon$	Final stabilizing solution

4. Analysis of Convergence and Theoretical Guarantees

The convergence analysis is grounded in domain invariance, monotonicity, and boundedness of correction sequences. Key statements:

• Domain Invariance: If $R_{22}(t) > 0$, the iterates and auxiliary solutions remain within the domain of well-posedness for the generator $\mathcal{G}$.
• Auxiliary Existence: If the linear part is stochastically stabilizable/detectable and $M - L R^{-1} L^\top \succ 0$, each inner Riccati subproblem admits a unique $T$-periodic stabilizing solution.
• Monotonicity: If the outer iterate yields a stable closed-loop, then the new inner solution $\widetilde{Y}_{K,W}$ dominates the previous iterate; thus, $$X^{(0)} \le X^{(1)} \le \cdots \le \widetilde{Y}_{K,W}$$.
• Global Convergence: Under the above structural assumptions, the sequences are globally convergent:
  • $\lim_{h \to \infty} Z^{(h)} = 0$,
  • The limit $X^*(t) = \lim_{h \to \infty} X^{(h)}(t)$ is the unique $T$-periodic stabilizing solution of the original SGTRDE.

The proof combines contraction arguments for auxiliary Riccati flows and monotone operator theory in the space of periodic symmetric matrix-valued functions.

5. Empirical Performance and Numerical Evaluation

Large-scale Monte Carlo experiments validate the iteration framework. For system orders $n = 1, \dots, 20$, 1,000 random trials per $n$ (totaling 20,000) are conducted. Data is generated with:

• $A_k \sim \mathcal{N}(0,1)$,
• $B_{0i} = 3I \pm 0.5H_i$,
• $B_{ki} \sim U[0,0.01]$,
• $R_{11} = -4I - U_{11} U_{11}^\top$,
• $R_{22} = 5I + U_{22} U_{22}^\top$,
• $L, M$ chosen to ensure $M - L R^{-1} L^\top \succ 0$,
• Fixed period $T=1$, with MATLAB default random seed.

Key observations:

• The required number of outer iterations for convergence (tolerance $1\mathrm{e}{-8}$) is typically $8$–$13$, exceeding $13$ in only $2$ out of $20,000$ trials.
• Inner iteration counts per outer step increase as the algorithm approaches stationarity.
• Low-dimensional systems exhibit higher variability in inner iteration counts, whereas higher-dimensional cases stabilize rapidly.

Qualitative histograms demonstrate consistent convergence behavior across dimensions and random instances, supporting the practical robustness of the dual-layer iteration scheme.

6. Applications and Broader Implications

Solutions to periodic SGTRDEs underpin feedback synthesis for zero-sum stochastic games with periodic coefficients, such as those encountered in financial engineering, signal processing, and systems with seasonal or cyclic behaviors. The presented algorithm provides a unified, numerically stable framework applicable to a broad class of such problems without restrictive simplifications or ad hoc regularization. The explicit dual-layer structure allows decomposition into tractable, definite-sign Riccati subproblems at each step, facilitating both theoretical analysis and scalable implementation.

A plausible implication is that the generality of this framework enables systematic studies and controller synthesis for new subclasses of time-periodic stochastic control problems, potentially extending to more complex multi-agent or non-zero-sum games, given further investigation of analogous structural properties.