Exploratory Mean-Variance Optimization

• Exploratory Mean-Variance (EMV) is a framework that replaces deterministic controls with a policy density to jointly optimize exploitation and exploration.
• It employs entropy regularization, typically via a Gaussian distribution, to explicitly trade off expected terminal wealth against risk and exploration.
• Extensions to continuous, discrete, and multi-asset models demonstrate EMV’s practical benefits in robust, risk-sensitive portfolio allocation.

Exploratory Mean-Variance (EMV) is an entropy-regularized, distributional-control reformulation of mean-variance optimization in which the decision variable is a policy density over actions rather than a single deterministic control. In portfolio problems, EMV retains the classical trade-off between expected terminal wealth and terminal variance, but it augments that criterion with an explicit exploration term—most commonly accumulated differential entropy—so that exploration and exploitation are optimized jointly within stochastic control or reinforcement learning. In the continuous-time formulations developed for portfolio selection, the optimal exploratory policy is Gaussian with time-decaying variance; later work extended the framework to large-scale multi-asset allocation, discrete-time portfolio models, finite-horizon Markov decision processes, regime-switching markets, jump-diffusions, robust drift-uncertainty settings, path-dependent controls, and Choquet-regularized exploration (Wang, 2019).

1. Conceptual foundations

The classical continuous-time mean-variance problem seeks

$$\min_u \operatorname{Var}[x_T^u]\qquad \text{subject to}\qquad \mathbb{E}[x_T^u]=z,$$

or, after Lagrangian embedding,

$\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$

EMV preserves this mean-variance target but replaces the point control $u_t$ by a randomized control law $\pi_t$, typically a density over portfolio allocations. Exploration is then measured by an accumulated regularizer, most prominently

$$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$

so the optimizer trades off terminal variance reduction against controlled policy dispersion.

This reformulation does not eliminate the foundational issues of mean-variance analysis. The mean-variance criterion remains time-inconsistent in its standard dynamic form, so many EMV solutions are pre-committed rather than dynamically time-consistent. More broadly, the classical literature on mean-variance and expected utility emphasizes that a two-dimensional $(\mu,\sigma)$ representation is coherent only under specific restrictions, such as quadratic utility or normal/CARA environments, and Borch’s paradox targets unrestricted mean-variance indifference reasoning (Johnstone et al., 2013). EMV therefore extends a particular mean-variance decision paradigm; it does not replace the broader expected-utility critique of that paradigm.

A second conceptual point is that, in EMV, randomization is not merely a numerical device for learning. In mean-variance control and mean-variance MDPs, randomization and history dependence can enlarge the attainable mean-variance region. Finite-horizon MDP results show that randomized or history-based policies can improve performance, and that randomization can be intrinsic to variance control rather than an auxiliary exploration heuristic (Mannor et al., 2011).

2. Continuous-time core formulation

In the large-scale continuous-time portfolio formulation, there is one riskless asset with constant rate $r$ and $d$ risky assets following geometric Brownian motions. With discounted dollar holdings $u_t\in\mathbb R^d$, the classical discounted wealth satisfies

$$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$

where $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$0 is the market price of risk defined by $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$1. EMV replaces $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$2 by a density $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$3 over $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$4, producing the relaxed state dynamics

$\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$5

For fixed Lagrange multiplier $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$6, the exploratory objective is

$\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$7

with $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$8 the temperature or exploration weight. The associated HJB equation minimizes, at each $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$9, an integral of the form

$u_t$0

Solving this inner variational problem yields a Gaussian feedback density

$u_t$1

In the closed-form solution, the optimal policy becomes

$u_t$2

and the optimal value function is explicit. The covariance schedule

$u_t$3

is highest at $u_t$4 and decreases in time, so exploration is front-loaded and attenuates as maturity approaches. The corresponding Lagrange multiplier satisfying $u_t$5 is

$u_t$6

These properties—closed-form value, Gaussian optimality, and time-decaying exploration—define the canonical continuous-time EMV structure (Wang, 2019).

3. Structural properties and relation to classical mean-variance

A central structural result is the separation between exploitation and exploration. In the continuous-time single-asset formulation, the optimal exploratory policy is

$u_t$7

so the Gaussian mean coincides exactly with the classical pre-committed mean-variance feedback control, while the Gaussian variance is a pure exploration term (Wang et al., 2019). In the $u_t$8-asset formulation, the same pattern appears: exploitation enters through the mean $u_t$9, whereas exploration enters through the covariance $\pi_t$0 (Wang, 2019).

The classical and exploratory problems are linked by a zero-temperature limit. As $\pi_t$1,

$\pi_t$2

and the EMV value converges to the classical mean-variance value. This establishes solvability equivalence between the classical and entropy-regularized formulations in the continuous-time models (Wang et al., 2019).

The multi-asset time-varying extension preserves the Gaussian structure but replaces constant coefficients by $\pi_t$3, $\pi_t$4, and the profitability functional

$\pi_t$5

The optimal exploratory portfolio distribution is

$\pi_t$6

so exploration expands in directions determined by $\pi_t$7 and scales with $\pi_t$8 (Li et al., 12 May 2025).

Discrete-time EMV yields an analogous but not identical structure. With wealth recursion $\pi_t$9, the optimal feedback density is Gaussian type,

$$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$0

with variance decaying geometrically as the horizon shortens (Cui et al., 2023). A common misconception is that Gaussian optimality in EMV requires Gaussian return distributions. In this discrete-time model, normality of returns is explicitly not required for optimality; only the first two moments $$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$1 and independence between $$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$2 and $$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$3 are used (Cui et al., 2023).

4. Discrete-time, MDP, and algorithmic formulations

The policy improvement theorem is one of the defining algorithmic mechanisms in EMV. In continuous time, if an admissible feedback policy $$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$4 has value $$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$5 with $$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$6, the Gaussian improvement map

$$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$7

satisfies

$$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$8

and the Gaussian family is closed under policy improvement (Wang, 2019). This theorem underlies practical RL schemes based on value fitting and policy updates rather than direct search over unrestricted control laws.

In large-scale multi-asset learning, one implementation minimizes an empirical squared Bellman error for a parametric value $$\mathcal H(\pi):=-\int_0^T\int \pi_t(u)\ln \pi_t(u)\,du\,dt,$$9 and Gaussian policy $(\mu,\sigma)$0, then applies the policy-improvement map using $(\mu,\sigma)$1 and $(\mu,\sigma)$2. In the time-varying multi-asset SAC adaptation, model parameters influencing the optimal policy are partitioned into three parts and learned progressively: per-asset $(\mu,\sigma)$3, cross-sectional $(\mu,\sigma)$4, and long-term profitability $(\mu,\sigma)$5. This partition is reported to reduce numerical instability and improve learning accuracy in the multi-asset case (Li et al., 12 May 2025).

The discrete-time RL formulation uses a Bellman-error objective

$(\mu,\sigma)$6

with a quadratic-form value parameterization and Gaussian actor consistent with the analytic solution. The target mean constraint is enforced by the stochastic-approximation update

$(\mu,\sigma)$7

or a sample-average variant for stability (Cui et al., 2023).

Finite-horizon MDP work generalizes mean-variance optimization beyond portfolio diffusion models. The formulation

$(\mu,\sigma)$8

fails to satisfy the standard dynamic programming principle because variance is non-additive. The remedy is a bilevel reduction based on the pseudo mean $(\mu,\sigma)$9 and pseudo variance $r$0, together with an augmented state $r$1. The resulting inner problem is a standard finite-horizon MDP on $r$2, and the original problem’s optimal policy is generally history-dependent deterministic. That paper does not explicitly use the term “Exploratory Mean-Variance (EMV),” but it describes a model-based foundation for EMV-style RL in finite horizon by separating exploration over pseudo means from policy improvement in the inner problem (Xia et al., 30 Jul 2025).

This MDP line also clarifies computational difficulty. In finite-horizon mean-variance MDPs, randomization and history dependence can strictly improve achievable mean-variance trade-offs, but computing a policy that maximizes the mean reward under a variance constraint is NP-hard for some policy classes and strongly NP-hard for others (Mannor et al., 2011).

5. Major extensions and specialized variants

A regime-switching extension, termed EMVRS, places the market in a finite-state Markov chain $r$3 with regime-dependent drift, volatility, and Sharpe ratio. The optimal value function is

$r$4

and the optimal exploratory control is Gaussian,

$r$5

A policy improvement theorem remains valid, but the paper argues that temporal-difference learning is not adequate in this setting; instead it introduces Orthogonality Condition learning based on the martingale property of the value-plus-entropy process $r$6 (Chen et al., 28 Jan 2025).

Jump models retain the exploratory-control logic while altering the state noise structure. In the entropy-regularized jump-diffusion EMV framework, the continuous-time exploratory wealth dynamics arise as the weak limit of natural discrete-time randomized controls. The optimal control distribution is again Gaussian, with mean and covariance

$r$7

and the optimal wealth process solves a linear SDE with an explicit stochastic-exponential representation (Bender et al., 2023). A separate jump-diffusion line treats the original mean-variance objective as time-inconsistent control and derives an exploratory equilibrium policy

$r$8

centered on the classical equilibrium control rather than the pre-committed optimum (Chen et al., 10 Dec 2025).

Robust EMV introduces model ambiguity in the drift through uncertainty in the market price of risk $r$9. The robust problem is a min-max EMV objective, and the worst-case choice is the admissible Sharpe ratio with minimal $d$0 norm: $d$1 The corresponding exploratory policy remains Gaussian,

$d$2

and the paper interprets robustness as weighting more on exploitation rather than exploration (Mou et al., 2021).

Two further extensions modify the exploration regularizer itself. Path-dependent EMV conditions the policy on the full wealth path $d$3 and uses functional Itô calculus, leading to a Gaussian policy on holdings

$d$4

and a path-dependent HJB on path space (Fang, 2023). Choquet-regularized EMV replaces entropy by a Choquet regularizer $d$5; the optimal exploratory distributions remain a location-scale family with quantile

$d$6

so the sampler shape depends on $d$7, generating exponential, uniform, or Gaussian exploratory laws (Guo et al., 2023).

6. Empirical performance, misconceptions, and limitations

Large-scale empirical studies consistently report that EMV is competitive in high-dimensional allocation. In S&P 500 experiments with monthly rebalancing, the large-scale continuous-time EMV method reported, for example, $d$8, $d$9, annualized return $u_t\in\mathbb R^d$0, Sharpe $u_t\in\mathbb R^d$1, and training time $u_t\in\mathbb R^d$2 hours, while the DDPG benchmark reported unannualized return $u_t\in\mathbb R^d$3, Sharpe $u_t\in\mathbb R^d$4, and training time $u_t\in\mathbb R^d$5 hours. For $u_t\in\mathbb R^d$6, EMV reported $u_t\in\mathbb R^d$7 and Sharpe $u_t\in\mathbb R^d$8; in daily rebalancing experiments, $u_t\in\mathbb R^d$9 reported $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$0 and Sharpe $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$1. Across experiments, the paper states that EMV consistently achieves $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$2 annualized returns and outperforms econometric baselines and deep RL by large margins (Wang, 2019).

Time-varying multi-asset SAC results are also favorable. On real-market datasets 29DJI, 57NASDAQ, and 340SP, the reported test metrics show SAC as the best method on MEAN, CEQ, SR, CEQ_TR, and SR_TR. For 340SP, the reported testing values are MEAN $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$3, CEQ $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$4, SR $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$5, CEQ_TR $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$6, and SR_TR $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$7, while the plug-in benchmark reports CEQ_TR $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$8 and SR_TR $$dx_t^u=\sigma u_t\cdot(\rho\,dt+dW_t),\qquad x_0^u=x_0,$$9 (Li et al., 12 May 2025).

Regime-switching and equilibrium-jump studies report gains from specialized learning rules. In EMVRS, OC learning dominates TD learning in simulation and real data; for example, with action constraint $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$00 and no short selling, EMVRS with OC learning reports annualized mean $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$01, volatility $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$02, and Sharpe ratio $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$03, while EMV reports $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$04, $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$05, and $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$06 (Chen et al., 28 Jan 2025). In the equilibrium jump-diffusion framework, the RL model is reported as profitable in 13 out of 14 tests on real market data (Chen et al., 10 Dec 2025).

Several misconceptions recur in discussions of EMV. EMV does not, by itself, resolve the time inconsistency of mean-variance optimization: pre-committed continuous-time formulations remain time-inconsistent, and equilibrium approaches are a separate line of development (Wang, 2019). Nor is EMV synonymous with entropy regularization alone: Choquet-regularized formulations produce non-Gaussian exploratory samplers such as exponential and uniform laws (Guo et al., 2023). Finally, superior empirical performance does not remove the model restrictions emphasized in the literature. Many theoretical EMV models assume geometric Brownian motion, constant risk-free rates, non-degenerate covariance, and no transaction costs; transaction costs and leverage constraints are often introduced only empirically. Reported limitations include model misspecification, nonstationarity, jumps, path dependence, sensitivity to $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$07 and $\min_u \ \mathbb{E}[(x_T^u-w)^2]-(w-z)^2.$08, multiple local optima in nonlinear MDPs, and the computational burden of augmented-state or path-dependent formulations (Xia et al., 30 Jul 2025).

Taken together, these results place EMV at the intersection of mean-variance control, entropy-regularized RL, and distributional policy design. Its defining structural feature is the replacement of deterministic feedback by analytically tractable exploratory distributions—most often Gaussian, but not necessarily so—whose mean preserves the classical exploitation direction and whose dispersion operationalizes exploration under an explicitly risk-sensitive criterion.