Stochastic Maximum Principle

• Stochastic Maximum Principle is a framework establishing necessary conditions for optimal control in stochastic systems by extending Pontryagin’s principle to infinite-dimensional spaces.
• It systematically handles unbounded boundary terms in SPDEs using operator-theoretic methods, linking control inputs and stochastic disturbances.
• The approach utilizes spike variations and variational inequalities, relying on enhanced regularity of the adjoint BSDE to ensure well-posed duality formulations.

The stochastic maximum principle (SMP) establishes necessary conditions for the optimality of controls in stochastic systems, extending Pontryagin’s classical principle to stochastic partial differential equations (SPDEs), especially in infinite-dimensional Hilbert space settings. When the control and noise enter via the boundary (as in boundary-controlled SPDEs), the SMP provides a dual-system characterization involving both the forward (state) SPDE and an infinite-dimensional backward stochastic differential equation (BSDE) for the adjoint process. The regularity of the solution to the adjoint equation is essential for the formulation and application of the maximum principle in this context.

1. Problem Formulation: SPDEs with Boundary Control and Boundary Noise

The system under consideration is an SPDE with both control and stochastic disturbance acting through the boundary, recast into the evolution equation form in a Hilbert space $H$. The state equation is: $$dX_t = [A X_t + F(t, X_t)]\,dt + (\lambda-A) D u_t\,dt + (\lambda-A) D_1\,dW_t + G(t, X_t)\,d\widetilde{W}_t,\quad X_0 = x$$

• $A$: unbounded operator generating an analytic $C_0$-semigroup $\{e^{tA}\}$ on $H$.
• $u_t$: control process, representing a boundary actuation, “lifted” to $H$ by $D$.
• $D$, $D_1$: boundary control/noise lift operators (mapping into $H$ or the appropriate fractional domain of $(\lambda - A)$).
• $W_t$, $\widetilde{W}_t$: independent Brownian motions.
• $G$, $F$: nonlinearities, with possible time and state dependence.

This formulation admits unbounded boundary terms which naturally arise in systems like heat equations with boundary actuation.

2. Stochastic Maximum Principle: Hamiltonian and Variational Inequalities

The SMP provides necessary conditions for optimality by formulating a comparison between the Hamiltonian under the current control and any perturbed admissible control. The functional to be minimized typically involves a running cost $l(t, x, u)$ and a terminal cost $h(x)$: $$J(u) = \mathbb{E} \left[ \int_0^T l(t, X_t, u_t) dt + h(X_T) \right]$$ The Hamiltonian is defined as: $$H(t, x, u, p) = \left( D^*[(\lambda-A)^* p], u \right)_U - l(t, x, u)$$ where:

• $p$ denotes the adjoint (costate) variable in $H$.
• $D^*[(\lambda-A)^*p]$: the dual action mapping (cf. application of the boundary control operator on the adjoint).

The SMP for nonconvex control domains states that, for optimal pair $(X, u)$,

$$H(t, X_t, u_t, Y_t) \geq H(t, X_t, v, Y_t),\quad \forall v \in U_{\text{ad}}$$

almost everywhere in $t \in [0,T]$, $P$-almost surely, where $U_{\text{ad}}$ is the set of admissible controls.

For convex $U_{\text{ad}}$, this is equivalent to the variational inequality: $$\langle H_u(t, X_t, \bar{u}_t, Y_t), v- \bar{u}_t \rangle_U \leq 0,\quad \text{for all } v \in U_{\text{ad}}$$ This condition expresses the first order necessary optimality, i.e., no infinitesimal change of the control can lower the cost.

3. The Adjoint Backward Stochastic Evolution Equation

A key technical component of the SMP is the construction and regularity analysis of the adjoint BSDE in the Hilbert space $H$. The adjoint equation is: $$-dY_t = \left[ A^*Y_t + F_x(t, X_t)^* Y_t + G_x(t, X_t)^* Z_t - l_x(t, X_t, u_t) \right] dt - Z_t\,dW_t - \widetilde{Z}_t\,d\widetilde{W}_t,\quad Y_T = -h_x(X_T)$$ where:

• $A^*$: adjoint of the generator.
• $F_x$, $G_x$: Fréchet derivatives with respect to the state variable.
• $Y_t \in H$: adjoint process (co-state).
• $Z_t$, $\widetilde{Z}_t$: “martingale” parts; these handle the randomness in the backward equation.

The well-posedness in infinite dimensions requires showing existence, uniqueness, and, crucially, extra regularity: $Y_t$ must belong to the domain of $D^*[(\lambda-A)^*]$ for all $t < T$ almost surely. This regularity is not automatic and is established through detailed functional analysis based on the properties of $A$, the semigroup, and the structure of $F$, $G$, $l$, $h$.

Without this property, the duality pairing $(D^*[(\lambda-A)^* Y_t], u)$ that appears in the Hamiltonian may not be defined, and thus the variational formulation of the SMP would be ill-posed.

4. Derivation of the SMP: Spike Variation and Regularity Usage

The derivation proceeds by the spike variation method:

1. Perturb the control $u_t$ on a small interval $[t_0, t_0 + \varepsilon]$ by an alternative value $v$.
2. Compute the first order variation in both the state $X_t$ and cost functional $J(u)$; evaluate using the regularity properties of both $X_t$ and $Y_t$.
3. Use Itô’s formula, integrating by parts, and exploiting the fact that $Y_t \in \mathrm{Dom}(D^*[(\lambda-A)^*])$, to rigorously obtain the variational inequality in terms of the Hamiltonian.

These steps yield: $$H(t, X_t, u_t, Y_t) \geq H(t, X_t, v, Y_t),\quad \text{a.e. } t \in [0, T],\quad P\text{-a.s.}$$ or, for convex $U_{\text{ad}}$,

$$\langle D^*[(\lambda-A)^* Y_t] - l_u(t, X_t, u_t), v-u_t \rangle_U \leq 0$$

This expresses the inability of any first-order control variation to decrease the cost, given the dual (adjoint) process.

5. Evolution System Perspective and Operator Framework

Formulating the problem as an evolution system in Hilbert space $H$ is critical to the analysis:

• The state equation uses the mild (semigroup) form, exploiting the analytic semigroup $e^{tA}$ generated by $A$.
• The unbounded boundary control and noise terms are handled by inclusion of $(\lambda - A) D$ and $(\lambda - A) D_1$ in the evolution.
• The abstract operator-theoretic formulation not only handles PDEs with boundary control but also unifies distributed and boundary noise, allowing direct extension to stochastic control problems in fluid dynamics, engineering, and finance.

The analysis demands that all regularity requirements are met, including domains of fractional and dual operators, to ensure the duality pairings in the SMP are mathematically justified.

6. Key Equations and Their Interpretations

Below is a table summarizing the key structures:

Component	Expression	Notes
State SPDE	$$dX_t = [A X_t + F(t,X_t)] dt + (\lambda−A)D u_t dt + (\lambda−A)D_1 dW_t + G(t,X_t)d\widetilde{W}_t$$	$A$ unbounded, $D$ lifts boundary control, unbounded terms allowed
Adjoint BSDE	$$-dY_t = [A^*Y_t + F_x^* Y_t + G_x^* Z_t - l_x] dt - Z_t dW_t - \widetilde{Z}_t d\widetilde{W}_t$$	Regularity: $Y_t \in \mathrm{Dom}(D^*[(\lambda-A)^*])$ a.s. required
Hamiltonian	$$H(t, x, u, p) = (D^*[(\lambda-A)^*p], u)_U - l(t, x, u)$$	Appears in variational inequality
Variational Inequality	$H(t, X_t, u_t, Y_t) \geq H(t, X_t, v, Y_t)$	Necessary condition for nonconvex controls

7. Implications and Generalizations

The developed SMP subsumes classical finite-dimensional results and enables rigorous treatment of stochastic control for boundary-actuated PDEs with possibly unbounded control operators—a setting common in applied problems. The work demonstrates that:

• Infinite-dimensional stochastic control requires regularity theory tied to operator domains.
• Duality between the forward state evolution and backward adjoint equation is central: the adjoint provides sensitivity information that is essential in the optimality condition.
• The established framework is crucial for the analysis and design of controllers in infinite-dimensional noisy systems, such as temperature regulation via boundary heaters in heat equation models, as well as in financial engineering and fluid dynamics.

The construction of the adjoint process with extra regularity is indispensable both for theoretical completeness and for practical computation or further paper of singular or boundary-constrained stochastic control systems.