Hamilton–Jacobi–Bellman QVI Overview

• HJB-QVI is a framework where dynamic programming couples HJB operators with intervention obstacles to model impulse control problems.
• It unifies continuous diffusion processes with discrete jumps, ensuring uniqueness through viscosity and modified supersolution formulations.
• Numerical schemes require monotone, stable, and nonlocally consistent discretizations, with policy iteration methods supporting robust convergence.

Hamilton–Jacobi–Bellman quasi-variational inequalities (HJB-QVIs) are dynamic-programming equations for control problems in which a continuation decision governed by an HJB operator is coupled to an intervention decision represented by a nonlocal obstacle. In a standard finite-horizon second-order form, the equation is

$$\begin{cases} \displaystyle \min \left ( - \sup_{b \in B} \left\{ \frac{\partial u}{\partial t} + L_b u + f(\cdot, b) \right\},\, u - \mathcal{M}u \right ) = 0, & \text{on } [0,T)\times \mathbb{R}^d, \ u(T,\cdot)=\max\bigl(g,\mathcal{M}u(T,\cdot)\bigr), & \text{on } \mathbb{R}^d, \end{cases}$$

where the intervention operator is

$$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$

The obstacle is “quasi-variational” because it depends on the unknown solution itself rather than on a fixed barrier (Azimzadeh et al., 2017). Closely related first-order and game-theoretic variants arise in optimal impulse control and in zero-sum games with impulse controls on one or both sides (Zhou et al., 2020, Cosso, 2012).

1. Canonical forms and nonlocal structure

A basic first-order HJB-type QVI for finite-horizon impulse control is

$$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$

with terminal condition

$V(T,x)=h(x),$

and nonlocal intervention operator

$$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$

where $K\subset \mathbb R^n$ is a closed convex cone of admissible impulses, $l(t,\xi)$ is the impulse cost, and $H(t,x,p)$ is the HJB Hamiltonian (Zhou et al., 2020). In this formulation, the equation compares the value of continuing under the Hamiltonian dynamics with the value of immediate intervention through $N[V]$.

In the second-order stochastic-control setting, the local operator is written through

$$L_b u(t,x) = \langle \mu(x,b), D_x u(t,x)\rangle +\frac12 \operatorname{tr}\!\bigl(\sigma\sigma^\top(x,b)\, D_x^2 u(t,x)\bigr),$$

so that the continuation branch becomes a fully nonlinear HJB term, while the intervention branch remains nonlocal through $$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$0 (Azimzadeh et al., 2017). The equation therefore has two coupled mechanisms: diffusion-and-control continuation, and instantaneous jumps.

For two-player zero-sum stochastic differential games in which both players use impulse controls, the associated Hamilton–Jacobi–Bellman–Isaacs equation is a double-obstacle QVI:

$$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$1

with terminal condition $$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$2, where

$$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$3

The obstacles are implicit because both depend on $$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$4 itself (Cosso, 2012).

An infinite-horizon deterministic two-player variant leads to a classic double-obstacle HJBI-QVI with impulse operators

$$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$5

and equation

$$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$6

This formulation isolates the continuous flow, the maximizer’s intervention option, and the minimizer’s intervention option within a single viscosity framework (Asri et al., 2021).

2. Viscosity formulations, obstacle constraints, and comparison

For HJB equations, the standard viscosity definitions compare smooth test functions with the solution at local maxima or minima. A natural extension of this idea to the QVI

$$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$7

gives the obvious subsolution and supersolution notions. However, the natural supersolution definition is too weak for HJB-type QVIs arising from optimal impulse control: at a touching point it permits

$$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$8

so a viscosity supersolution in that sense need not satisfy the obstacle constraint $$\mathcal{M}u(t,x) \coloneqq \sup_{z\in Z(t,x)} \Bigl\{ u\bigl(t,x+\Gamma(t,x,z)\bigr)+K(t,x,z) \Bigr\}.$$9 (Zhou et al., 2020).

This distinction is central to the comparison problem. For HJB equations, comparison between sub- and supersolutions is standard. For the QVI, the literature can still yield uniqueness of viscosity solutions, but the comparison between an arbitrary subsolution and an arbitrary supersolution need not follow from the natural QVI supersolution notion. The failure is therefore not a failure of uniqueness itself; it is a failure of the natural supersolution notion to encode the obstacle constraint strongly enough (Zhou et al., 2020).

A modified supersolution definition restores the expected structure. A continuous $$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$0 is required to satisfy:

1. $$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$1,
2. $$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$2 on $$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$3,
3. whenever $$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$4 attains a local minimum at $$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$5 with

$$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$6

one has

$$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$7

The PDE inequality is imposed only where the obstacle is strictly inactive. Under assumptions $$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$8–$$\min \Big\{ V_t(t,x)+H\bigl(t,x,V_x(t,x)\bigr),\; N[V](t,x)-V(t,x)\Big\}=0, \qquad (t,x)\in [0,T)\times \mathbb R^n,$$9 and the stated growth and regularity conditions, Theorem 3.1 gives

$V(T,x)=h(x),$0

for a subsolution $V(T,x)=h(x),$1 and a modified supersolution $V(T,x)=h(x),$2, thereby restoring the comparison principle (Zhou et al., 2020).

For the double-obstacle stochastic game, the paper uses the standard Crandall–Ishii–Lions notion. If $V(T,x)=h(x),$3 is a viscosity subsolution and $V(T,x)=h(x),$4 is a viscosity supersolution of the HJBI-QVI, both uniformly continuous on $V(T,x)=h(x),$5, then $V(T,x)=h(x),$6 on $V(T,x)=h(x),$7. This yields uniqueness in the class of uniformly continuous functions (Cosso, 2012).

In the deterministic infinite-horizon game, uniqueness of the classic double-obstacle QVI is not guaranteed under the weaker assumptions. The remedy is a new HJBI QVI in which the maximizer obstacle is replaced by the local gradient constraint

$V(T,x)=h(x),$8

so that the equation becomes

$V(T,x)=h(x),$9

Under the proportionality assumption $$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$0, this reformulation admits a comparison principle and hence a unique bounded continuous viscosity solution (Asri et al., 2021).

3. Dynamic programming and control-theoretic origins

The HJB-QVI is the PDE expression of dynamic programming for problems in which intervention is discrete and state-changing. In the optimal impulse-control problem revisited in the appendix of (Zhou et al., 2020), the controlled state satisfies

$$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$1

with impulse control

$$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$2

and cost functional

$$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$3

The value function is the infimum of $$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$4, and under the assumptions it satisfies the HJB-QVI

$$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$5

With the modified supersolution definition, the value function is the unique viscosity solution (Zhou et al., 2020).

For the finite-horizon stochastic differential game with impulse controls on both sides, the lower and upper values are defined by Elliott–Kalton strategies:

$$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$6

The dynamic programming principle is proved for both value functions, and the analysis shows that $$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$7 and $$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$8 are viscosity solutions of the same double-obstacle HJBI-QVI. By uniqueness, they coincide, so the game admits a value (Cosso, 2012).

An infinite-horizon deterministic analogue is studied in (Asri et al., 2021). There the lower and upper values are

$$N[V](t,x)=\inf_{\xi\in K}\Big(V(t,x+\xi)+l(t,\xi)\Big),$$9

and the dynamic programming principle yields the corresponding HJBI-QVI. Under the proportionality property assumption on the maximizer cost, the lower and upper value functions solve the new HJBI QVI uniquely, so they coincide and the game has a value.

A recurrent structural feature is that the continuation region is defined by strict separation from the intervention branch. In the finite-time horizon stochastic-and-impulse setting, the continuation set is

$K\subset \mathbb R^n$0

which excludes the degenerate case of intervening everywhere (Ieda, 2013). This separation between continuation and intervention is precisely what the QVI formalizes.

4. Numerical discretization and convergence theory

A central numerical theme is the construction of monotone implicit schemes that preserve the comparison structure of the nonlocal problem. For the finite-horizon HJB-QVI associated with combined stochastic and impulse control, an implicit finite-difference discretization is

$K\subset \mathbb R^n$1

with terminal condition $K\subset \mathbb R^n$2 and boundary condition $K\subset \mathbb R^n$3 on the truncated boundary (Ieda, 2013). The method is implicit in the PDE sense, and its stability proof is based on the algebraic structure of the discrete matrix rather than on a CFL-type time-step restriction. The resulting bound $K\subset \mathbb R^n$4 is obtained with a constant $K\subset \mathbb R^n$5 independent of $K\subset \mathbb R^n$6 and $K\subset \mathbb R^n$7 (Ieda, 2013).

A more general convergence theory for implicit finite-difference schemes is developed in (Azimzadeh et al., 2017). The key notion is nonlocal consistency, introduced because standard PDE consistency is insufficient when the limiting equation depends on a nonlocal operator acting on the approximate solution. For every smooth $K\subset \mathbb R^n$8, every bounded family $K\subset \mathbb R^n$9, and every $l(t,\xi)$0, the scheme must satisfy the limsup/liminf relations

$l(t,\xi)$1

and

$l(t,\xi)$2

Combined with monotonicity, stability, and a comparison principle for the nonlocal viscosity problem, this yields a Barles–Souganidis-type theorem:

$l(t,\xi)$3

The paper proves local uniform convergence for both a penalty scheme and a semi-Lagrangian scheme, the latter when $l(t,\xi)$4 is control-independent (Azimzadeh et al., 2017).

The numerical analysis also identifies structural requirements that are specific to HJB-QVIs. The intervention operator must be discretized with monotone interpolation; higher-order interpolants can violate monotonicity and nonlocal consistency. In higher dimensions, wide-stencil discretizations are required for diffusion terms if monotonicity is to be preserved. For bounded computational domains, fixed truncation can cause overstepping error in the semi-Lagrangian method; the stated remedy is to refine the mesh near the boundary so that boundary spacing shrinks appropriately (Azimzadeh et al., 2017).

A recent implicit HJB-QVI scheme for optimal market making with an alpha signal verifies monotonicity, stability, and consistency and applies the comparison principle to conclude local uniform convergence to the unique viscosity solution (Meteykin, 24 Dec 2025). The paper rewrites each time-step problem as

$l(t,\xi)$5

and solves it by policy iteration. The required matrix conditions are verified using the finite policy set, boundedness of $l(t,\xi)$6 and $l(t,\xi)$7, a connectivity condition in the impulse graph, and the fact that $l(t,\xi)$8 and $l(t,\xi)$9 are $H(t,x,p)$0-matrices with suitable diagonal dominance. The policy iteration sequence is nondecreasing and converges to the unique discrete solution, in finitely many steps if the policy set is finite (Meteykin, 24 Dec 2025).

5. Policy iteration, Newton structure, and quadrature consistency

Policy iteration is frequently interpreted as a Newton-type or semismooth Newton method. For the semilinear second-order HJB problem

$H(t,x,p)$1

with

$H(t,x,p)$2

the continuous policy iteration selects

$H(t,x,p)$3

and then solves

$H(t,x,p)$4

Under the stated assumptions, the sequence is monotonically globally convergent in $H(t,x,p)$5, and convergence is asymptotically superlinear unless the exact solution is reached in finitely many steps (Hall et al., 23 Jun 2026).

At the discrete level, exact integration preserves the algebraic equivalence between the residual and nonresidual policy updates. With exact integration and exact arithmetic, the identity

$H(t,x,p)$6

ensures that the residual and nonresidual formulations generate the same iterates, and the discrete iterates inherit local superlinear convergence under the stated hypotheses and uniform stability of the linearized problems (Hall et al., 23 Jun 2026).

The practical difficulty is quadrature. In modern form-based finite element software, the stiffness matrix and right-hand side may be compiled and integrated separately, and the library may choose distinct quadratures automatically. The paper defines matching quadrature by

$H(t,x,p)$7

and nonmatching quadrature when these sets differ. If quadrature is matching, the quadrature-based residual and nonresidual formulations remain equivalent. If quadrature is nonmatching, the equivalence fails; the two implementations no longer solve the same discrete nonlinear problem and may even produce different sequences of iterates (Hall et al., 23 Jun 2026).

The mechanism of convergence loss is explicit. Policy iteration depends on an exact cancellation between the linearized operator and the residual at the previous iterate. If quadrature differs between the terms, the discretized Hamiltonian on the right-hand side is not evaluated with the same numerical approximation as the terms used in the linearized bilinear form. The update is then no longer the exact Newton/policy step for the discretized nonlinear operator. The paper remarks that the residual form under nonmatching quadrature resembles an inexact Newton method: it still converges, but only linearly. The nonresidual form can fail to converge altogether (Hall et al., 23 Jun 2026).

This observation is stated for HJB equations, but the paper also identifies its direct relevance to HJB-QVI numerical schemes. Many HJB-QVI methods rely on policy iteration or Newton-like iterations applied to variational formulations with obstacle or intervention terms. If different parts of the discretized operator are assembled with different quadrature rules, the algebraic identity underlying the residual update can break, which can destroy the expected superlinear behavior or even convergence. Enforcing common quadrature for all terms tied to the nonlinear Hamiltonian or intervention operator is therefore presented as a simple and effective safeguard (Hall et al., 23 Jun 2026).

6. Structural variants and application domains

The HJB-QVI framework appears in several application classes without changing its core interpretation as a comparison between continuation and immediate intervention. In optimal forest harvesting over a finite time horizon, the value function satisfies

$H(t,x,p)$8

with terminal condition

$H(t,x,p)$9

The numerical results show a threshold-type optimal harvesting policy: for $N[V]$0 and sufficiently large $N[V]$1, the finite-horizon strategy switch point approaches the infinite-horizon threshold, while close to $N[V]$2 the threshold becomes much larger, so harvesting is delayed as terminal time approaches (Ieda, 2013).

In optimal market making with an alpha signal, the controlled state is

$N[V]$3

and the value function is reduced through

$N[V]$4

The reduced value function solves an HJB-QVI whose first branch optimizes the continuation dynamics under limit-order controls and whose second branch optimizes the gain from immediate market-order impulses (Meteykin, 24 Dec 2025). The numerical results show that near $N[V]$5 the market maker posts both bid and ask limit orders; as $N[V]$6 becomes strongly positive or strongly negative, the strategy concentrates on one side; and for large $N[V]$7 the maker may combine limit and market orders (Meteykin, 24 Dec 2025).

Game-theoretic variants broaden the structure from single-obstacle to double-obstacle QVIs. In the finite-horizon stochastic differential game with impulse controls on both sides, the two intervention operators $N[V]$8 and $N[V]$9 represent the best immediate interventions available to the two players, and the resulting QVI characterizes the unique viscosity value of the game (Cosso, 2012). In the deterministic infinite-horizon zero-sum game, the new HJBI QVI replaces one nonlocal obstacle by a local gradient constraint under the proportionality property assumption on the maximizer cost, making comparison and uniqueness available in a bounded continuous class (Asri et al., 2021).

A common misconception is that all viscosity formulations or all discretizations that look formally similar are interchangeable. The cited results show otherwise. In the analytical theory, the natural supersolution definition for an HJB-type QVI can be too weak to support comparison unless the obstacle constraint is imposed appropriately (Zhou et al., 2020). In the numerical theory, monotonicity, stability, nonlocal consistency, and comparison are the structural ingredients behind convergence (Azimzadeh et al., 2017), and quadrature inconsistency can invalidate the Newton or policy-iteration mechanism even when the continuous theory predicts superlinear convergence (Hall et al., 23 Jun 2026). These results together place the HJB-QVI at the intersection of viscosity theory, dynamic programming, nonlocal analysis, and structure-preserving numerical approximation.