Entropy-Regularized Mean-Variance Objective

• The entropy-regularized mean-variance objective is a continuous-time stochastic control framework that augments terminal quadratic loss with a Shannon-entropy penalty to encourage exploration and robustness.
• It replaces deterministic controls with probability density functions, resulting in a Gaussian optimal policy where variance quantifies risk and exploration across different settings.
• This framework finds applications in portfolio optimization, electricity bidding, and robust order execution, and yields closed-form value functions via HJB equations.

import requests, textwrap, re, xml.etree.ElementTree as ET, json, math url='http://export.arxiv.org/api/query?search_query=all:%22entropy-regularized%20mean-variance%22&start=0&max_results=5' try: r=requests.get(url, timeout=20) print(r.status_code) print(r.text[:500]) except Exception as e: print("ERR", e)

to=python code to=python code to=python code to=python code The entropy-regularized mean-variance objective is a continuous-time stochastic control criterion that augments a mean-variance terminal objective with an entropy term on randomized controls. In the portfolio formulations considered in "Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework" (Wang et al., 2019), "Entropy Regularization as Robustness under Bayesian Drift Uncertainty" (Au, 18 Feb 2026), and "Electricity Virtual Bidding Strategy Via Entropy-Regularized Stochastic Control Method" (Fang, 2023), the control at each time is not a point action but a probability density over holdings or allocations, and the objective combines terminal quadratic loss with either a Shannon-entropy penalty of the form $\int \pi \ln \pi$ or, equivalently, an entropy reward written through $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$.

1. Canonical objective and control-theoretic formulation

In the basic continuous-time mean-variance formulation, discounted wealth is driven by a risky allocation and Brownian noise. For a single risky asset, the classical controlled wealth SDE is

$$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$

where $\rho=(\mu-r)/\sigma$ is the Sharpe ratio and $u_t$ is the discounted dollar amount invested in the risky asset. The exploratory version replaces the point control $u_t$ by a density $\pi_t(\cdot)$ on $\mathbb R$, with relaxed dynamics determined by the mean and variance of the sampled action (Wang et al., 2019).

The associated entropy-regularized mean-variance criterion is written with a Lagrange multiplier $w$ as

$J[\pi;w] = \E\bigl[(X_T^\pi-w)^2\bigr] + \lambda\,\E\!\Bigl[\int_0^T\!\!\int_{\mathbb R}\pi_t(u)\ln\pi_t(u)\,du\,dt\Bigr] -(w-z)^2,$

where $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$0 is the target expected terminal wealth and $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$1 is the exploration weight. The first term is the squared-error around $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$2, while the entropy term rewards higher-entropy policies because $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$3 is the negative of differential Shannon entropy (Wang et al., 2019).

A multidimensional version appears in electricity virtual bidding. There, wealth evolves as

$H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$4

where the drift vector $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$5 and volatility matrix $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$6 are functions of observed meteorological variables $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$7. The virtual bidding problem is then transformed into a mean-variance portfolio management problem and approached by the exploratory mean-variance portfolio management framework (Fang, 2023).

Under Bayesian drift uncertainty, the same objective is embedded in partial information. The discounted wealth process satisfies

$H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$8

with posterior mean and variance $H(\pi_t)=-\int \pi_t(u)\ln\pi_t(u)\,du$9 obtained from the Kalman-Bucy filter. The entropy-regularized cost is

$$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$0

and this is stated to be dual to

$$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$1

with suitable matching of $$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$2 and $$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$3 (Au, 18 Feb 2026).

2. HJB structure and the Gaussian optimal policy

The defining analytical feature of the entropy-regularized mean-variance objective is that dynamic programming produces an HJB equation whose inner optimization is over probability densities rather than point controls. In the single-asset case, the value function

$$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$4

satisfies

$$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$5

with terminal condition $$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$6 (Wang et al., 2019).

The stationarity condition in $$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$7 yields a Gibbs density,

$$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$8

and, because the exponent is quadratic in $$dX_t^u=\sigma\,u_t\,(\rho\,dt+dW_t),\qquad X_0^u=x_0,$$9, the optimizer is Gaussian. For the known-drift single-asset problem, the explicit solution is

$\rho=(\mu-r)/\sigma$0

so the optimal feedback policy is Gaussian with time-decaying variance (Wang et al., 2019).

The same structure persists in multidimensional and partial-information settings. In the exploratory mean-variance portfolio problem used for virtual bidding, the optimizer is

$\rho=(\mu-r)/\sigma$1

so both the mean and covariance inherit the market-price parameters $\rho=(\mu-r)/\sigma$2 and $\rho=(\mu-r)/\sigma$3 (Fang, 2023).

With Bayesian drift uncertainty, the HJB is written in $\rho=(\mu-r)/\sigma$4 and the inner minimization depends on

$\rho=(\mu-r)/\sigma$5

The optimizer is again Gaussian,

$\rho=(\mu-r)/\sigma$6

with the mixed derivative $\rho=(\mu-r)/\sigma$7 coupling learning into the mean (Au, 18 Feb 2026).

3. Explicit value functions and the link to classical mean-variance control

A central result in the literature is that entropy regularization preserves the tractability of continuous-time mean-variance control. In the single-asset RL formulation, the value function admits the quadratic representation

$\rho=(\mu-r)/\sigma$8

which gives $\rho=(\mu-r)/\sigma$9 and directly produces the Gaussian optimizer (Wang et al., 2019).

The zero-entropy limit connects the exploratory and classical problems. As $u_t$0, the optimal distribution $u_t$1 converges weakly to the Dirac mass at the classical optimal control

$u_t$2

and the value function converges pointwise to

$u_t$3

The exploratory problem with $u_t$4 and the classical problem with $u_t$5 are stated to share the same Lagrange multiplier $u_t$6, the same drift in the optimal wealth SDE, and solvability equivalence (Wang et al., 2019).

The virtual bidding formulation yields the same qualitative link. The mean of the optimal Gaussian policy is exactly the classical mean-variance optimal holding, proportional to $u_t$7, but shrunk by the risk-adjusted term $u_t$8. Its covariance is proportional to $u_t$9 and acts as an exploration bonus of order $u_t$0. As $u_t$1, the policy collapses to the deterministic classical policy; for $u_t$2, the policy remains Gaussian and explores around the mean (Fang, 2023).

Under Bayesian drift uncertainty, the value function remains quadratic in wealth:

$u_t$3

with explicit closed-form coefficients $u_t$4 and $u_t$5 along the deterministic path of $u_t$6. The paper emphasizes that $u_t$7 is independent of $u_t$8 and $u_t$9, so the entropy term enters through the additive entropy-premium term and the policy variance rather than through the quadratic wealth coefficient itself (Au, 18 Feb 2026).

4. Exploration, robustness, and information masking

The interpretation of the entropy term depends on the informational structure of the model. In the reinforcement-learning formulation with known drift, the entropy-regularized mean-variance criterion is explicitly presented as a device for achieving the best tradeoff between exploration and exploitation. The policy variance

$\pi_t(\cdot)$0

decays exponentially from an initially high exploration level, while the mean equals the classical exploitative control (Wang et al., 2019).

Under Bayesian drift uncertainty, the same formal regularizer acquires a different meaning. The mean control is exactly the deterministic Bayesian Markowitz feedback and is independent of $\pi_t(\cdot)$1, whereas the policy variance depends only on $\pi_t(\cdot)$2 and $\pi_t(\cdot)$3. Because filtering $\pi_t(\cdot)$4 is independent of the policy, the extra variance does not speed up learning; instead it provides belief-dependent robustness. The optimal variance is symmetric in $\pi_t(\cdot)$5, minimized at $\pi_t(\cdot)$6, and strictly increasing in $\pi_t(\cdot)$7, so action randomization increases when the certainty-equivalent mean position is most aggressive (Au, 18 Feb 2026).

A third interpretation appears in mean-variance Stackelberg competition with asymmetric information. There the leader maximizes

$\pi_t(\cdot)$8

and the optimal randomized trading policy is Gaussian with variance

$\pi_t(\cdot)$9

Larger $\mathbb R$0 produces a wider Gaussian and more masking of the true information from the follower, whereas larger $\mathbb R$1 or larger $\mathbb R$2 reduce randomization (Huang et al., 3 Sep 2025).

These formulations rule out a common simplification: entropy regularization is not uniformly an exploration mechanism. In the RL setting it is tied to exploration; in the Bayesian partial-information setting it is explicitly interpreted as robustness; and in the Stackelberg setting it is used to prevent information leakage (Wang et al., 2019, Au, 18 Feb 2026, Huang et al., 3 Sep 2025).

5. Variants and application domains

The objective has been embedded in several distinct stochastic environments. The common elements are randomized controls, mean-variance terminal criteria, and an entropy or relative-entropy regularizer; the differences lie in the state dynamics, the source of uncertainty, and the operational role of randomization.

Setting	Dynamics and uncertainty	Role of entropy term
Electricity virtual bidding	Brownian motion with drift; drift and volatility depend on meteorological variables	Exploration around mean-variance-optimal holding
Portfolio optimization with jumps	Lévy-driven return process and exploratory wealth SDE with compensated jump measure	Gaussian control under jump-diffusion dynamics
Bayesian drift uncertainty	Kalman-Bucy filtering for unknown Sharpe ratio	Belief-dependent robustness
Stackelberg asymmetric information	Leader randomizes actions observed by follower	Information masking
Robust optimal order execution	KL penalty relative to prior market-rate distribution	Distributional robustness

In electricity virtual bidding, price differences between the day-ahead market and the real-time market are modeled as Brownian motion with drift, where the drift rate and volatility are functions of meteorological variables. The paper states that one estimates $\mathbb R$3, for example by regression on temperature, wind, and humidity, and $\mathbb R$4, for example via a rolling-window covariance from historical LMP-differences; these quantities then enter both the drift term of $\mathbb R$5 and the shape of the exploration bonus (Fang, 2023).

In the jump setting, the discounted return process is Lévy-driven, the exploratory wealth SDE is obtained as the weak-convergence limit of a natural discrete-time randomization scheme, and the continuous-time HJB again yields a Gaussian optimal distribution. The coefficient fields $\mathbb R$6 and $\mathbb R$7 are defined through a backward PIDE, and the optimal covariance is $\mathbb R$8, while the optimal wealth process admits a closed-form representation via the Doléans-Dade exponential (Bender et al., 2023).

Relative-entropy regularization in optimal order execution is closely related but not identical. There the inner minimization is over distributions $\mathbb R$9 for the market trading rate, and the objective adds a Kullback-Leibler term

$w$0

Under the LQG ansatz and a Gaussian prior, the posterior $w$1 is Gaussian, the HJI reduces to coupled Riccati and linear ODEs, and finite $w$2 forces the posterior to stay close to the prior, guarding against misspecified market-rate models (Wang et al., 2023).

6. Algorithmic consequences and recurrent points of confusion

The entropy-regularized mean-variance objective is not only analytically tractable; it also supports policy-improvement and learning procedures. In the RL formulation, if a smooth feedback policy $w$3 has value $w$4 with $w$5, then the update

$w$6

satisfies

$w$7

Iterating this improve-evaluate procedure is stated to converge, in finitely many steps for the mean-variance LQ structure, to the global optimum $w$8 and $w$9 (Wang et al., 2019).

The implementable EMV algorithm parameterizes $J[\pi;w] = \E\bigl[(X_T^\pi-w)^2\bigr] + \lambda\,\E\!\Bigl[\int_0^T\!\!\int_{\mathbb R}\pi_t(u)\ln\pi_t(u)\,du\,dt\Bigr] -(w-z)^2,$0 by the quadratic/exponential form implied by the HJB solution and $J[\pi;w] = \E\bigl[(X_T^\pi-w)^2\bigr] + \lambda\,\E\!\Bigl[\int_0^T\!\!\int_{\mathbb R}\pi_t(u)\ln\pi_t(u)\,du\,dt\Bigr] -(w-z)^2,$1 as a Gaussian family, computes the continuous-time TD error

$J[\pi;w] = \E\bigl[(X_T^\pi-w)^2\bigr] + \lambda\,\E\!\Bigl[\int_0^T\!\!\int_{\mathbb R}\pi_t(u)\ln\pi_t(u)\,du\,dt\Bigr] -(w-z)^2,$2

updates $J[\pi;w] = \E\bigl[(X_T^\pi-w)^2\bigr] + \lambda\,\E\!\Bigl[\int_0^T\!\!\int_{\mathbb R}\pi_t(u)\ln\pi_t(u)\,du\,dt\Bigr] -(w-z)^2,$3 and $J[\pi;w] = \E\bigl[(X_T^\pi-w)^2\bigr] + \lambda\,\E\!\Bigl[\int_0^T\!\!\int_{\mathbb R}\pi_t(u)\ln\pi_t(u)\,du\,dt\Bigr] -(w-z)^2,$4 by stochastic gradient descent to minimize squared TD error, and updates the Lagrange multiplier through the Robbins-Monro rule

$J[\pi;w] = \E\bigl[(X_T^\pi-w)^2\bigr] + \lambda\,\E\!\Bigl[\int_0^T\!\!\int_{\mathbb R}\pi_t(u)\ln\pi_t(u)\,du\,dt\Bigr] -(w-z)^2,$5

The paper reports that this algorithm outperforms both an adaptive control based method and a deep neural networks based algorithm in simulations (Wang et al., 2019).

Several misconceptions are resolved directly by the cited formulations. One is that entropy regularization necessarily changes the deterministic feedback law; in the Bayesian partial-information model, the mean control is identical to deterministic Bayesian Markowitz feedback and only the policy variance changes (Au, 18 Feb 2026). Another is that entropy-regularized mean-variance control must be diffusion-only; the jump formulation shows that Gaussian randomized controls persist in a Lévy environment, although the coefficient fields are determined by a backward PIDE rather than an ODE system (Bender et al., 2023). A further misconception is that entropy regularization always operates through the agent’s own action distribution; the order-execution model uses relative entropy with respect to a prior on the market trading-rate distribution, yielding a robust stochastic differential game rather than a purely exploratory control problem (Wang et al., 2023).

Across these formulations, the entropy-regularized mean-variance objective retains the mean-variance target-tracking structure, admits HJB-based closed forms in LQ and related settings, and replaces deterministic controls by Gaussian policies whose variance quantifies exploration, robustness, or strategic masking according to the surrounding information model (Wang et al., 2019).