Discrete-Time Backward Stochastic LQ Control Problem

Abstract: This paper focuses on the discrete-time backward stochastic linear quadratic (BSLQ) optimal control problem with nonhomogeneous system terms and cost function cross terms. The terminal constraint of such systems distinguishes it from forward stochastic systems, posing unique challenges for analysis and solution. Within the Hilbert space framework, we first clarify the necessary and sufficient conditions for problem solvability, then introduce the backward stochastic system maximum principle to derive the Hamiltonian system characterizing the optimal control. After equivalent transformation of the original problem, we use the decoupling method to obtain the corresponding Riccati equation, and present the explicit state feedback expression of the optimal control and the analytical form of the value function. Finally, numerical examples verify the effectiveness and feasibility of the proposed method. The innovation lies in expanding the model generality: addressing the structural asymmetry issue in the Riccati equation with cross-term cost functions, we propose system equivalent transformation and decoupling techniques. Our theoretical results provide a new analytical framework for dynamic optimization problems such as financial portfolio optimization and risk management.

• The paper develops an explicit optimal feedback law by reformulating the backward stochastic LQ control problem using Hilbert space techniques.
• It introduces a decoupling transformation and backward Riccati recursion to handle nonhomogeneities and cross terms in the cost function.
• Numerical results validate the theoretical framework, confirming unique semi-positive Riccati solutions and well-defined value functions.

Discrete-Time Backward Stochastic Linear-Quadratic Optimal Control: Theory, Solution Methods, and Numerical Implementation

Problem Setting and Motivation

The paper addresses the discrete-time backward stochastic linear-quadratic (BSLQ) optimal control problem in the general setting of nonhomogeneous system terms and cost function cross terms. Such structure arises in practical scenarios like financial portfolio optimization and risk management, where system dynamics are often governed by terminal constraints, only tractably modeled by backward stochastic difference equations (BS$\Delta$Es). Unlike the extensively studied continuous-time BSLQ problems, the discrete-time analog is less explored, with significant technical obstacles introduced by the presence of nonhomogeneities and cross terms in both the dynamics and the quadratic cost.

Formally, the authors consider a controlled, finite-horizon, BS$\Delta$E system with conditional expectations capturing the martingale increments due to underlying stochasticity. The cost functional is quadratic but features nontrivial coupling and cross terms ($S_k$ matrices and inhomogeneous processes $\eta_k, \rho_k$), contrasting with the standard quadratic cost. The paper’s main objective is to characterize the solvability, derive the explicit optimal control policy, and demonstrate the results with constructive numerical examples.

Hilbert Space Framework and Structural Analysis

A key methodological contribution is the reformulation of the control problem in a Hilbert space operator framework, providing a rigorous setting for deriving optimality conditions, continuity, differentiability, and convexity properties of the cost functional. By exploiting the linear structure of BS$\Delta$Es, the authors decompose the solution into terminal-state-driven, control-driven, and inhomogeneous components. This enables the use of bounded linear operators ($\mathcal{A}, \mathcal{B}, \mathcal{C}$) to rewrite the cost functional in a generalized quadratic form.

Under mild definiteness and regularity assumptions on the system matrices (uniform positive definiteness of $R_k$, nonnegativity of $Q_k - S_k^\top R_k^{-1} S_k$, and $G_0 \geq 0$), strict convexity, coercivity, and continuity of the cost are established. The necessary and sufficient conditions for optimality follow from direct Hilbert space analysis: Fréchet/Gâteaux differentiability leads to a stationarity system, and the uniqueness of the optimum is guaranteed by strict convexity.

Solution via Stochastic Maximum Principle and Riccati Decoupling

The central technical development is the derivation of the coupled stochastic maximum principle for this backward setting. This involves a system of forward-backward stochastic difference equations (FBS$\Delta$Es) for the state and an associated adjoint process, as well as a Hamiltonian stationarity condition. The Hamiltonian incorporates all cross and inhomogeneous terms, and the optimality condition reduces to a coupled algebraic equation at each time step involving the state, adjoint, and control.

An important and challenging aspect is the structural asymmetry introduced in the Riccati recursion by the cross terms—unlike the symmetric Riccati equations of classical (forward) discrete-time LQ control, the backward setting with cross-coupling requires a nontrivial equivalent transformation. The authors introduce a system-equivalent transformation based on auxiliary matrix sequences $\Delta$0, which absorb the cross-terms and reformulate the original BSLQ into an equivalent problem with a Riccati recursion admitting a unique, positive semidefinite solution. The transformation also reparametrizes the cost and state equations to expose the decoupling structure.

The Riccati difference equations, defined backward from the terminal time, allow for explicit construction of both the feedback gain sequence and the inhomogeneous offset term in the optimal control law. The resulting optimal control is affine in the state and terminal data, thus implementable in feedback form.

Explicit State-Feedback Formulation

The culmination of the theoretical analysis is the explicit feedback law for the optimal control: $\Delta$1 where $\Delta$2 is the solution of the adjoint/dual process, itself coupled to the forward/backward state via the decoupled Riccati equations. The "decoupling field" is characterized by a sequence $\Delta$3 solving the aforementioned Riccati difference equation, and a backward process $\Delta$4 determined by a secondary backward stochastic difference equation, both constructed under the Hilbert space framework.

The value function admits an explicit, analytical form in terms of these Riccati solutions and the backward process, allowing not only for practical evaluation but also for a sharp theoretical understanding of cost-to-go as a function of terminal state and system parameters.

Numerical Implementation and Results

To validate the theory, the authors implement their solution approach for a four-period ($\Delta$5) case study. All system and cost matrices, as well as terminal constraint, are explicitly specified. A full computation of the sequence $\Delta$6 (for the equivalent transformation), Riccati solutions $\Delta$7, the backward process $\Delta$8, and the resulting feedback gains $\Delta$9 and offsets $S_k$0 is performed using exact backward and forward recursions.

The computed optimal value function $S_k$1, as well as all matrix and vector sequences, are presented explicitly. The results confirm that the Riccati equation solution is uniquely semi-positive definite, the feedback law is implementable, and the value function is well-defined and consistent with the theoretical predictions. The numerical example demonstrates the structural tractability of the solution, even for systems with nonhomogeneity and cross-coupling.

Implications and Future Directions

The methodological innovations in this work—namely the equivalent system transformation, operator Hilbert space framing, and the backward Riccati decoupling—provide a unified and tractable approach for discrete-time BSLQ control with general structure. The ability to accommodate nonhomogeneous system terms and cost cross terms directly addresses practical requirements in discrete-time controlled systems (e.g., asset-liability management in finance, risk-sensitive control).

From a theoretical perspective, the operator-based formulation is extensible to more complex classes of controlled backward SDEs, mean-field interactions, and potentially to discrete stochastic differential games. On the practical side, the explicit feedback and value function forms enable offline precomputation and online implementation for systems actuated in discrete time steps, which is particularly relevant for digital control platforms and financial trading systems.

A promising extension, as identified in the paper, is the generalization to discrete-time zero-sum Stackelberg stochastic linear-quadratic games and systems with additional constraints, which would require further expansion of the Riccati-based framework to saddle-point and hierarchical control settings.

Conclusion

The paper develops a comprehensive theoretical and algorithmic framework for discrete-time backward stochastic LQ control problems with nonhomogeneous system terms and cost cross terms. Through operator Hilbert space analysis, transformation techniques, and Riccati decoupling, the authors derive explicit feedback control laws and value functions, providing both theoretical guarantees and practical computational recipes. The results fill a key gap in discrete-time backward stochastic control and lay the foundation for further advances in the discrete stochastic optimal control of constrained, cross-coupled systems (2604.10510).