Linear error bounds for HJB equations in finite horizon control problems

Abstract: We study semi Lagrangian approximation schemes for Hamilton Jacobi Bellman equations arising from finite horizon optimal control problems. Classical error estimates for these schemes include the term $\frac{1}{Δt}$ which leads to pessimistic convergence bounds and is not observed in numerical experiments. In this work, we provide improved error estimates under standard regularity assumptions on the dynamics, the running cost, and the final cost, assuming the presence of a positive discount factor. The new bound depends linearly on the time step, the spatial mesh size, and a measure of the temporal oscillation of the control, thus removing the mixed term appearing in previous analyses. The proof relies on a refined comparison between continuous and discrete cost functionals and on stability estimates for the controlled dynamics. Numerical experiments confirm first-order convergence in both space and time and suggest that the improved behavior persists even in the undiscounted case.

• The paper proves the linear bound v-V C( t+ x+M_u) for fully discrete semi-Lagrangian schemes, eliminating the classical time-step degeneracy.
• The analysis combines a discrete dynamic-programming representation, Euler trajectory stability, and cost-consistency estimates, requiring > L_I and Lipschitz dependence on controls.
• Numerical tests achieve nearly first-order convergence, with experimental orders of 0.9979 for a discounted LQR problem and 1.0050 for an undiscounted two-dimensional test, suggesting linear accuracy may persist when =0.

Motivation and context

Semi-Lagrangian (SL) schemes are a standard tool for approximating Hamilton–Jacobi–Bellman (HJB) equations arising from finite horizon optimal control, since they are monotone, stable without CFL-type restrictions, and yield feedback controls. The classical error estimate for the fully discrete SL scheme of Falcone and Giorgi gives

$$\|v - V\|_{\infty} \leq \tilde{C}\left(\frac{\Delta x}{\sqrt{\Delta t}} + \sqrt{\Delta t}\right),$$

which contains a mixed space–time term that degenerates as $\Delta t \to 0$ for fixed $\Delta x$. This pessimistic bound is not observed numerically. For infinite horizon discounted problems, sharper first-order estimates have been established by exploiting contractivity induced by the discount factor; Alla and Mayer extend that analysis to the finite horizon setting, where the explicit time dependence of the value function complicates the argument (2602.15215).

Setting and discretization

The paper considers the controlled ODE $\dot y = f(y,t,u)$ on $[t_0,T]$ with cost functional

$$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$

under the standard assumptions: $f$ and $g$ bounded, continuous, and uniformly Lipschitz in all three arguments (state, time, control); $\psi$ Lipschitz and bounded. The discount factor satisfies $\lambda > 0$. The dynamics is discretized by explicit Euler on a uniform temporal grid, the cost integral by a right-endpoint quadrature with discrete discounting $\Delta t \to 0$0, and the value function by the fully discrete SL scheme with multilinear interpolation $\Delta t \to 0$1 on a mesh of diameter $\Delta t \to 0$2 over a polytope $\Delta t \to 0$3 satisfying a dynamics-invariance condition.

Main results

The analysis rests on two structural lemmas and culminates in a linear error bound.

Discrete dynamic programming representation. A first theorem shows that the numerical solution $\Delta t \to 0$4 coincides exactly with the infimum, over discrete control sequences, of a fully discrete cost functional built with $\Delta t \to 0$5. This mirrors the continuous definition of the value function and is the key device allowing a direct comparison of continuous and discrete optima rather than a comparison-principle argument.

Trajectory stability. Under hypotheses (H1)–(H3), the continuous trajectory and the piecewise-constant extension of its Euler approximation satisfy, via a Gronwall argument,

$\Delta t \to 0$6

where $\Delta t \to 0$7 measures the temporal oscillation of the control, and $\Delta t \to 0$8 involves the interpolation constant.

Cost consistency. If $\Delta t \to 0$9 and $\Delta x$0, the difference between the continuous and discrete cost functionals along matched controls is bounded by $\Delta x$1. The proof splits the discrepancy into four terms: interpolation/state error ($\Delta x$2), discount-factor discretization ($\Delta x$3), initial-cell mismatch ($\Delta x$4), and terminal cost mismatch ($\Delta x$5). Notably, the term $\Delta x$6—the gap between $\Delta x$7 and $\Delta x$8—is handled via Lagrange's theorem and a careful bound on $\Delta x$9, which stays uniformly bounded on $\dot y = f(y,t,u)$0, yielding $\dot y = f(y,t,u)$1. The assumption $\dot y = f(y,t,u)$2 is essential here: it makes the exponential weight dominate the Gronwall-grown trajectory error so that the integrals remain uniformly bounded.

Main convergence theorem. Combining these ingredients, the authors obtain

$\dot y = f(y,t,u)$3

with $\dot y = f(y,t,u)$4 under regularity of the optimal controls (e.g., Lipschitz continuity in time). This removes the pathological $\dot y = f(y,t,u)$5 amplification entirely and establishes first order in both variables simultaneously. The proof compares the numerical optimum against the discrete cost of the true optimal control and vice versa, using Lemma 2 to close both directions. Two caveats deserve emphasis: the result requires the additional Lipschitz-in-control assumptions on $\dot y = f(y,t,u)$6 and $\dot y = f(y,t,u)$7 relative to Falcone–Giorgi's framework, and it requires a strictly positive discount satisfying $\dot y = f(y,t,u)$8, which couples the discount rate to the dimension-dependent interpolation Lipschitz constant.

Numerical validation

Two tests confirm the theory, both run with $\dot y = f(y,t,u)$9 and uniform refinement, measuring the relative $[t_0,T]$0 error against known exact value functions.

Test	Setting	Final EOC
Test 1: LQR-type problem	$[t_0,T]$1, $[t_0,T]$2, $[t_0,T]$3, $[t_0,T]$4	0.9979
Test 2: 2D undiscounted problem	$[t_0,T]$5, terminal cost only, $[t_0,T]$6	1.0050

Test 1 satisfies all theoretical hypotheses and exhibits EOC increasing toward 1 upon refinement (from 0.9833 at $[t_0,T]$7 to 0.9979 at $[t_0,T]$8). Test 2 deliberately violates the discount assumption ($[t_0,T]$9) yet still converges linearly, with EOC decreasing from 1.0385 to 1.0050. This second experiment constitutes empirical evidence—though no proof—that the linear bound persists without discounting, which is the most notable open claim in the paper.

Limitations and open questions

The theoretical framework has clear boundaries that the authors acknowledge implicitly through their hypotheses. First, the convergence theorem requires $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$0 with $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$1 growing with the spatial dimension $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$2 through the norm-equivalence and interpolation constants, so high-dimensional or weakly discounted problems may fall outside the proven regime even when numerics behave well. Second, the extra Lipschitz-in-control regularity on $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$3 and $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$4 strengthens the assumptions of the original Falcone–Giorgi analysis, so the improvement is not free. Third, the term $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$5 is only controlled if one can bound the temporal oscillation of (near-)optimal controls; the paper notes this holds for Lipschitz controls but does not address cases where optimal controls lack such regularity. Finally, whether the linear estimate holds for $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$6 remains unproven—the Test 2 evidence is suggestive but outside the theory—and extension to other discretization schemes and broader classes of control problems is left open.

Conclusion

This work transfers the improved, discount-driven error analysis from infinite horizon to finite horizon HJB equations, replacing the classical $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$7 bound with a clean linear estimate $$J(x,t,u)=\int_t^T g(y(s),s,u(s))\,e^{-\lambda(s-t)}ds + e^{-\lambda(T-t)}\psi(y(T)),$$8 under standard regularity plus a positive discount dominating the interpolation Lipschitz constant. The proof strategy—identifying the discrete scheme with the infimum of a fully discrete cost and comparing costs along matched controls—is elementary but effective. Numerical experiments confirm sharpness of the bound, and the observed convergence in the undiscounted case identifies the removal of the discount assumption as the natural next target for this line of analysis.