Extended Mean Field Control

• Extended Mean Field Control is a class of stochastic control problems that incorporates dependencies on both the state and the joint state-control distributions for enhanced modeling flexibility.
• It utilizes weak closed-loop formulations, a stochastic maximum principle, and reformulations via causal optimal transport to streamline analysis and computation.
• The framework supports practical applications in systemic risk, market microstructure, and networked systems, offering a robust bridge between theory and numerical implementation.

Extended Mean Field Control (MFC) refers to a class of stochastic control problems in which the cost function and the state dynamics depend not only on the state and its distribution but also on the joint distribution of the controlled state and the control process. This extension, compared to classical mean field control, incorporates richer forms of interaction by allowing the law of the control process to enter both the evolution equations and the objective, thus offering additional modeling flexibility—especially in systems where agents interact through both states and controls. The foundational framework and a comprehensive mathematical justification, including a stochastic maximum principle and transport-based variational methods, are established in "Extended Mean Field Control Problems: stochastic maximum principle and transport perspective" (Acciaio et al., 2018).

1. Problem Formulation and Weak Closed-Loop Controls

In the extended MFC setting, the admissible control $\alpha$ may in general depend on more than the current state, possibly on the whole path or on additional measurable structures. The dynamics of the controlled process $(X_t)_{t\in[0,T]}$ are of McKean–Vlasov type, written as: $$dX_t = b(t, X_t, \mu_t, \alpha_t)dt + \sigma(t, X_t, \mu_t, \alpha_t)dW_t,$$ where $\mu_t = \mathcal{L}(X_t, \alpha_t)$ denotes the joint law of state and control at time $t$, $b$ and $\sigma$ are measurable coefficients, and $W$ is a Brownian motion.

The cost functional takes the form: $$J(\alpha) = \mathbb{E}\left[\int_0^T f(t, X_t, \mu_t, \alpha_t)dt + g(X_T, \mu_T)\right],$$ where $f$ and $g$ are measurable functions, possibly convex in control and law arguments.

A central result is that, under appropriate convexity and monotonicity assumptions, any admissible control can be replaced by a weak closed-loop control measurable with respect to the state filtration, without increasing the cost. The construction leverages the following:

• The induced joint law $\pi$ on the canonical path space is causal and projects all involved processes onto a state-based filtration.
• By projecting the drift (control) process onto the state filtration and applying Lévy’s martingale representation theorem, one constructs a replacement control adapted only to the state history.
• Jensen’s inequality, together with the convexity of the running cost, is used to show that the new control yields a cost no higher than the original.

Therefore, optimal controls for the extended MFC problem can always be taken in (weak) closed-loop form: measurable with respect to the state (and possibly additional independent randomness for measurable selection).

2. Stochastic Maximum Principle for Extended MFC

The Pontryagin stochastic maximum principle (SMP) is extended to handle dependencies on the joint law of state and control. The necessary and sufficient conditions for optimality are derived in the weak formulation, ensuring that any optimal control must satisfy the following:

• There exists an adapted adjoint process (often a backward SDE) linked to the lifted Hamiltonian
• The control $\alpha_t$ at each time $t$ should minimize the Hamiltonian, which now depends not only on the current state, adjoint, and control, but also on the joint law $\mu_t$.

The rich structure, including the presence of joint distributions, is treated through variational calculus over the space of causal, measure-valued processes.

3. Optimal Transport Formulation and Discretization

A key methodological insight is the reformulation of the extended MFC problem in terms of causal optimal transport on the path space. Specifically:

• The weak formulation interprets the optimal control problem as an optimal transport problem constrained by causality—i.e., transport plans must be adapted to the natural time filtration.
• This reformulation suggests numerical discretization schemes in which the class of admissible controls is restricted to policies determined by the (discretized) state process, significantly simplifying the implementation.

4. Projection Argument and Cost Improvement

The argument for restricting to closed-loop controls (as given in (Acciaio et al., 2018)) is constructed as follows:

• Start with an arbitrary feasible tuple $(\Omega, \mathcal{F}, \mathbb{P}, W, X, \alpha)$.
• Construct the joint law $\pi$ on canonical path space and identify the "drift part" $\beta$ as the density of the underlying control process.
• Project $\beta$ onto the smaller filtration generated by the state, obtaining $\bar\beta$.
• Define the replacement control $u_t = b^{-1}(X_t, \cdot, \mu_t)(\bar\beta_t)$, which is a function of $X_t$ (and possibly measurable selection).
• Using properties of convex functions and Jensen's inequality:

$q(\mathbb{E}[\cdot]) \leq \mathbb{E}[q(\cdot)]$

it follows that replacing the drift with its projection does not increase the expected cost and can only improve it under strict convexity.

• The final step utilizes measurable selection theorems to ensure existence of a measurable function representing the closed-loop control.

This justifies, both from theoretical and practical perspectives, searching for optimal policies solely among weak closed-loop controls.

5. Implementation Considerations

For practitioners:

• When modeling extended MFC systems, it is sufficient to parameterize controls as functions of the state and (possibly) auxiliary randomness, even when the original control class is very general.
• Numerical and computational schemes can be specialized to closed-loop Markov policies, reducing complexity and facilitating discretization (e.g., via causal dynamic programming).
• The projection and replacement steps can be operationalized algorithmically by estimating the drift conditional on the state history and constructing feedback laws accordingly.

A trade-off is that the existence and uniqueness of such optimal weak closed-loop controls rely on the precise convexity and monotonicity structure of the problem data.

6. Significance and Theoretical Impact

The conclusion that optimality in extended MFC can always be achieved within the class of weak closed-loop controls provides a conceptual unification:

• It brings the extended theory in line with classical results for standard stochastic control and mean field control (when the cost and dynamics do not explicitly depend on the control law).
• It greatly facilitates analysis and computation since the class of controls is smaller, and measurable selection arguments ensure the representation can be made explicit for practical purposes.

This result is a critical bridge toward further developments in numerical approximation, transport-based formulations, and the stochastic maximum principle for modern high-dimensional MFC systems.

7. Future Directions and Related Developments

The projection principle for closed-loop optimality in extended MFC is foundational for several subsequent research directions:

• Discretization and computation through causal optimal transport on path space
• Generalizations to systems with singular controls or constraints
• Integration with modern reinforcement learning and deep learning-based control, leveraging the closed-loop representability
• Extension to settings involving common noise, nonlocal interactions, or multiple interacting populations (see (Bensoussan et al., 2018) for extensions to multipopulation settings).

The insights and methodologies developed in this context have significant relevance to applications in systemic risk, market microstructure, energy systems, and networked control.