Distribution-Steering Problems in Stochastic Control

• Distribution-steering problems involve designing control strategies that steer a state’s probability distribution from an initial to a target distribution rather than just tracking a trajectory.
• They utilize mathematical tools such as Riccati and Lyapunov equations alongside convex optimization approaches to address uncertainty in systems like stochastic robotics and quantum control.
• Feasibility conditions differ between finite-horizon and stationary cases, highlighting challenges in maintaining controllability and ensuring proper distribution shaping.

Distribution-steering problems concern the design of control strategies for dynamical systems so that the state probability distribution is transferred (or “steered”) from a prescribed initial distribution to a final, typically specified, distribution under dynamics that may be subject to noise or uncertainty. Unlike point-to-point or trajectory tracking in classical control, distribution steering aims to control the law of the process—often represented by its probability density function (PDF) or its moments, rather than just the mean state. Addressing such problems is fundamental in applications where uncertainty quantification, shape, or spread control of state ensembles is crucial, including molecular systems, stochastic robotics, microthermodynamics, and quantum systems.

1. Mathematical Formulation of Distribution-Steering

Consider a linear stochastic system

$$dx^u(t) = A(t)x^u(t)dt + B(t)u(t)dt + B_1(t)dw(t),\quad x^u(0) \sim \mathcal{N}(0, \Sigma_0)$$

and a target terminal distribution at $T$, $x^u(T) \sim \mathcal{N}(0, \Sigma_T)$. The control problem is to find an admissible control $u(t)$ minimizing

$J(u) = \int_0^T u(t)^\top u(t)\,dt,$

subject to the state evolving under the controlled process and ending with the prescribed terminal law.

This generalizes to nonlinear and discrete-time systems, and to settings with more general initial and terminal distributions. In the infinite-horizon (stationary) case, the problem is to maintain a prescribed stationary distribution under constant state-feedback.

Distribution steering is mathematically equivalent to solving a stochastic optimal mass transport (OMT) problem or a Schrödinger bridge (in the terminology of stochastic control and large deviations), particularly when distributions are Gaussian and the system is linear (Chen et al., 2014, Chen et al., 2016).

2. Optimality Conditions and Riccati/Schrödinger Systems

The finite-horizon distribution-steering problem admits optimality conditions that can be expressed through coupled matrix Riccati equations:

• The feedback law $u(x,t) = -B(t)^\top \Pi(t)x$, where $\Pi$ solves a backward Riccati differential equation,
• The evolution of the state covariance $\Sigma(t)$ is governed by a differential Lyapunov equation,
• Auxiliary equations for an additional variable $H$ allow the boundary conditions (on $\Sigma_0$ and $\Sigma_T$) to be met via the so-called Schrödinger system, a pair of coupled Riccati equations imposing the correct covariances at endpoints.

For the infinite-horizon (stationary) case, the feedback takes the form $u(t) = -Kx(t)$, with $K = B^\top \Pi$, where $\Pi$ solves the algebraic Riccati equation

$A^\top \Pi + \Pi A - \Pi BB^\top \Pi + Q = 0,$

and the stationary covariance $\Sigma$ must satisfy an algebraic Lyapunov equation. Uniqueness of $K$ is tied to the requirement that $A - BK$ is Hurwitz (stable) (Chen et al., 2014).

3. Feasibility and Controllability

Finite-Horizon Feasibility: If the pair $(A, B)$ is controllable, it is always possible to transfer the state covariance from $\Sigma_0$ to any other positive definite $\Sigma_T$ in time $T$, for any disturbance directionality (i.e., $BB^\top$ and $B_1B_1^\top$ may differ). This includes situations where the control and noise channels do not overlap.

Stationary Feasibility: Not all positive definite matrices $\Sigma$ can be realized as stationary covariances under constant state-feedback; admissible stationary covariances are those satisfying a Lyapunov-like algebraic constraint, typically requiring that $A\Sigma + \Sigma A^\top + B_1B_1^\top$ lies in the range of $B$ (Chen et al., 2014). If the column space of $B$ is contained in that of $B_1$, all such stationary problems are feasible with a Hurwitz feedback.

4. Solution Techniques and Convex Optimization

Direct solution of the coupled Riccati (Schrödinger) system is numerically challenging. Instead, both the finite- and infinite-horizon problems can be recast as convex optimization problems:

• The control design is reformulated in terms of a pair $(U(t), \Sigma(t))$, leading to a convex cost $$\int_0^T \operatorname{trace}(U^\top \Sigma^{-1} U)\,dt$$ with linear dynamics for $\Sigma(t)$,
• With appropriate time discretization, this yields an efficiently solvable semidefinite program (SDP),
• For the stationary problem, the cost $\operatorname{trace}(K\Sigma K^\top)$ is minimized subject to the algebraic Lyapunov constraint, again forming a convex SDP.

This approach scales to high dimensions and is particularly well suited for computational implementation (Chen et al., 2014).

5. Example: Inertial Particle Model and Non-Overlapping Channels

As a canonical case, the evolution of inertial particles (position and velocity) with the model

$dx = v\,dt + dw, \qquad dv = u\,dt$

leads to a situation with non-overlapping channels ($B = [0;1]$, $B_1 = [1;0]$). This system cannot be controlled in the “direct” channel in which noise enters. Nevertheless, the controllability condition is satisfied, and a stationary covariance (e.g., $\Sigma_1 = \left[\begin{smaLLMatrix}1&-1/2\-1/2&1/2\end{smaLLMatrix}\right]$) is achievable by solving an algebraic Lyapunov equation for the associated feedback gain. The transient steering from $\Sigma_0 = 2I$ to $\Sigma_1$ over a finite horizon is solved via the SDP. Once at $\Sigma_1$, the system is stabilized with the constant gain (Chen et al., 2014).

Time evolution plots clearly distinguish the steering phase (covariance and feedback gain transient) and the stationary regime (constant gain and stationary covariance).

6. Implications, Generalizations, and Limitations

Key Implications:

• For controllable systems, any prescribed Gaussian transfer (on the covariance) is possible on a finite horizon, extending the classic Schrödinger bridge and linear-quadratic regulator settings;
• The stationary setting is inherently more restrictive, as only admissible covariances—those satisfying the Lyapunov constraint—can be maintained;
• The convex SDP reformulation provides practical, scalable computation for high-dimensional steering tasks, bypassing the limitations of coupled Riccati integrations.

Potential Applications include precise temperature or uncertainty regulation in micromechanical systems, particle manipulation, and state shaping in quantum and stochastic systems where distributional (rather than mean) control is paramount.

Limitations involve the possible non-uniqueness of certain Riccati equation solutions, the challenge in ensuring existence and uniqueness for the full Schrödinger system, and the need for advanced numerical techniques in general (especially with time-varying or high-dimensional dynamics).

7. Summary Table: Core Ingredients and Properties

Aspect	Finite-Horizon Case	Stationary (Infinite-Horizon) Case
Distribution Type	Gaussian (arbitrary covariance)	Gaussian (subject to Lyapunov constraint)
Feasibility	Always (if (A,B) controllable)	Only for admissible covariances
Feedback Law	Time-varying state feedback	Constant gain state feedback
Riccati Equations	Coupled dynamic (Schrödinger system)	Algebraic Riccati/Lyapunov equations
Numerical Method	Convex SDP via (U(t), Σ(t)) variables	Convex SDP; algebraic constraints
Application Scenario	Steering + stationary maintenance	Maintaining statistical steady-state

This structure reflects the rigorous foundation for distribution-steering problems as advanced in minimum energy steering of linear stochastic systems, encompassing both foundational optimality results and practical, computational solution approaches (Chen et al., 2014).