Constrained Limit-Average Optimal Control

• Constrained limit-average problems are defined as optimizing long-term average cost under asymptotic constraints, critical for control and geometric optimization.
• The framework utilizes occupational measures and Carathéodory's theorem to reduce infinite-horizon problems to a finite mix of stationary and periodic solutions.
• Applications include Cheeger sets and PDE singular limits, demonstrating practical cyclic switching strategies for satisfying averaged constraints.

Constrained limit-average problems concern the minimization (or maximization) of a long-run average cost or reward under one or more averaged (asymptotic) constraints on system trajectories or policies. Such formulations appear across stochastic control, game theory, online learning, and geometric optimal control, providing a unifying lens to address performance under both objective and constraint satisfaction in the asymptotic regime.

1. Mathematical Formulation and Occupational Measure Framework

Consider a continuous-time control system on the plane with state $x(t)\in K\subset\mathbb{R}^2$, control $u(t)\in U\subset\mathbb{R}^d$, and dynamics

$\dot{x}(t) = f(x(t), u(t)).$

Let $L(x,u)$ denote the cost density and $g_i(x,u)$ ($i=1,\dots,m$) constraint densities. The weighted limit-average (or time-average) cost is

$$J(u) = \limsup_{T\to\infty} \frac{1}{T} \int_0^T L(x(t), u(t))\,dt.$$

Each constraint is enforced on time averages: $$\limsup_{T\to\infty} \frac{1}{T} \int_0^T g_i(x(t), u(t))\,dt \leq c_i.$$ Assume $K, U$ compact; $f$, $L$, $g_i$ continuous and $f$ Lipschitz in $x$; and uniform controllability (from any $x_1$ to $x_2$ in $K$ in bounded time). The admissible controls keep the state in $K$ for all $t \geq 0$.

To handle the infinite-horizon averaging, the problem is reformulated in terms of limiting occupational measures $$\mu \in \mathcal{M} \subset \mathcal{P}(K \times U)$$. For a sequence $T_n\to\infty$, and given trajectories $(x_n, u_n)$, the empirical measures

$$\mu_n(A) = \frac{1}{T_n} \mathrm{Leb}\{ t\in [0, T_n] : (x(t), u(t)) \in A \}$$

converge weakly to some limiting measure $\mu$. The set $\mathcal{M}$ is compact and convex. The average cost and constraints are expressed as $\mu(L) = \int L\,d\mu$ and $\mu(g_i) \leq c_i$. Thus, the constrained limit-average problem becomes

$$\inf_{\mu \in \mathcal{M},\; \mu(g_i) \le c_i} \mu(L).$$

This framework admits ratio objectives of the form $\mu(p) / \mu(q)$, with $q > 0$ everywhere, crucial for applications such as shape optimization (Bright, 2013).

2. Structure of Optimal Solutions: Poincaré–Bendixson Type Results

A foundational result is that, for planar systems under the above hypotheses, every extreme point of the set of achievable occupation measures is realized either by a stationary solution ($x(t) \equiv x_0$, $u(t) \equiv u_0$) or a periodic solution whose trace is a Jordan curve.

For unconstrained problems, the minimum of the limit-average (or ratio) cost is realized by a stationary or single periodic solution. If constraints are introduced, Carathéodory's theorem ensures that the optimal occupation measure $\mu^*$ is a convex combination of at most $m+1$ extreme points, corresponding to at most $m+1$ stationary or periodic orbits. The control can therefore be implemented by cyclic switching among these orbits according to the convex coefficients, ensuring the averaged constraints are satisfied. When $m=1$ and the constraint is cumulative (holds for all finite $T$), the switching schedule can enforce this strong constraint at all times, not just asymptotically (Bright, 2013).

3. Principal Lemmas and Proof Constructs

The existence theory and structure results hinge on several key lemmas:

• Compactness of the occupational measure set $\mathcal{M}$: follows from compactness of $K \times U$ and tightness criteria.
• Convexity under uniform controllability: if every $x_1$ to $x_2$ transition is possible, convex combinations of occupation measures associated to initial states/controls can also be realized.
• Carathéodory-type lemma: the intersection of $\mathcal{M}$ with $m$ constraint hyperplanes has extreme points that are convex combinations of at most $m+1$ extreme points of $\mathcal{M}$.
• Quasi-concavity of the ratio functional $V(\mu) = \mu(p)/\mu(q)$: on the convex set $\Delta = \{ \mu(g) : \mu \in \mathcal{M} \}$.

Together, these imply that limit-average optimal control in the plane reduces, in both constrained and unconstrained cases, to analysis of a finite number of stationary or periodic arcs.

4. Applications: Cheeger Sets and PDE Singular Limits

The theory directly connects to two prominent geometric and variational problems:

Cheeger sets: The Cheeger ratio $$v^*_C = \max_{E \subset \Omega} \mathrm{Area}(E) / \mathrm{Length}(\partial E)$$ can be phrased as

$$V(E) = \frac{1}{T} \int_0^T x^1(t) \dot{x}^2(t) dt,$$

where $x$ parameterizes $\partial E$ as a positively oriented Jordan curve with fixed-speed parametrization. This is a limit-average problem with no constraints, and the existence theory guarantees extremal solutions are single periodic (i.e., the boundary traces an optimally shaped Cheeger set) (Bright, 2013).

Generalized Cheeger sets: When area and perimeter are weighted, the same structural conclusion holds.

Singular limits in PDEs: The stationary states of the 1D van der Waals/Cahn–Hilliard functional,

$$E_\epsilon[u] = \int_0^1 \left(B(u) + \frac{1}{2}\epsilon^2 (u')^2 \right) dx, \quad \int_0^1 u = M,$$

reduce, as $\epsilon \to 0$, to a constrained limit-average problem for a 1D ODE with an averaged mass constraint. The minimizer cycles between stationary or periodic orbits, yielding a lamellar two-phase structure matching known singular patterns observed in this limit.

5. Significance, Limitations, and Further Directions

The occupational measure framework subsumes exact solution existence for both unconstrained and averaged-constrained planar limit-average optimal control, allowing a reduction from infinite-horizon behavior to a finite-dimensional, finitely-cyclic strategy (number of cycles at most $m+1$ for $m$ constraints). This framework bridges the gap between classical calculus of variations, optimal control, and geometric problems, yielding structural results (e.g., periodicity or cyclical switching for the optimal control law) that are not generally available in higher-dimensional or more complex systems.

Key limitations include the restriction to planar systems (for the periodic-or-stationary extreme point structure) and the need for compactness and controllability assumptions. Extensions to higher dimensions or non-convex control systems may not retain these properties. Further, while existence and structure results are comprehensive, deriving explicit switching schedules or trajectories in practice may require additional regularity or computational insights, especially as the number of constraints increases.

6. Summary Table: Core Theoretical Results

Regime	Existence Result	Structure of Solution
No constraints	Minimum at periodic or stationary orbit	Single cycle or stationary trajectory
$m$ constraints	Minimum is convex comb. of $\leq m+1$	Cyclic switching among at most $m+1$ orbits
$m=1$, cumulative	Strong constraint holds for all $T$	Schedule ensures constraint always met

This synthesis provides the existence theory, geometric structure, and practical realization of constrained limit-average control in the planar setting, serving as a foundation for further developments in infinite-horizon variational problems and geometric optimization (Bright, 2013).