Neural HJR: Hamilton–Jacobi Reachability Learning

• Neural HJR is a computational framework that uses neural networks to approximate Hamilton–Jacobi value functions, encoding backward reachable sets and safe control strategies.
• It leverages PDE residual minimization and structural priors to efficiently train models that satisfy safety semantics and feedback control laws in high-dimensional systems.
• Recent advancements extend Neural HJR to decentralized multi-agent planning, manifold-constrained control, and reinforcement learning, balancing theoretical convergence with empirical performance.

Neural Hamilton–Jacobi Reachability Learning (HJR) denotes a class of methods that represent a Hamilton–Jacobi or Hamilton–Jacobi–Isaacs reachability value function $V(t,x)$ with a neural model and train that model so that it satisfies the associated HJ/HJI equation, terminal or boundary condition, and, in some variants, additional physics, symmetry, or manifold structure. In these methods, level sets of $V$ encode backward reachable tubes or safe sets, while $\nabla_x V$ enters the Hamiltonian and induces safety-preserving feedback control. The line of work was crystallized by DeepReach’s neural PDE solver for high-dimensional reachability (Bansal et al., 2020), then extended with a uniform convergence theorem for a supremum-norm formulation (Hofgard, 2024), decentralized multi-agent motion planning (Chen et al., 18 Jul 2025), manifold-constrained planning (Chen et al., 5 Nov 2025), neural-operator surrogates (Li et al., 28 Apr 2025), and verification frameworks for learned reachability certificates (Yang et al., 2024, Smith et al., 26 Mar 2026).

1. Mathematical formulation and safety semantics

For controlled dynamics with disturbance,

$$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$

a standard reachability target set is encoded as $L=\{x:\ell(x)\le 0\}$. A common cost functional is

$$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$

and, for backward reachability with worst-case disturbance, the value function is

$$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$

Dynamic programming yields the HJI variational inequality

$$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$

with Hamiltonian

$$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$

Its zero sublevel set is the backward reachable tube: $\mathrm{BRT}(t)=\{x:V(t,x)\le 0\}.$ This is the canonical worst-case safety semantics used in neural HJR (Hofgard, 2024).

Several papers use the equivalent safety-oriented sign convention in which the safe set is a superlevel set. In decentralized multi-agent motion planning, the backward reachable tube PDE is written as

$V$0

with collision set $V$1, and the safe set is

$V$2

Under this convention, $V$3 means there exists a control strategy that keeps the system out of collision for all disturbance strategies over the horizon, and $V$4 means collision is inevitable in the worst case (Chen et al., 18 Jul 2025).

A distinct but mathematically aligned formulation connects HJ reachability to reinforcement learning. A travel-cost value function with a suitable running cost has the property that its negative sublevel set equals the strict backward-reachable tube, and fixed points of small-step RL value iteration converge to the viscosity solution of a forward discounted HJB equation (Solanki et al., 12 Jan 2026). This places neural HJR within a broader value-learning framework: PDE residual minimization, Bellman updates, and reachability-based RL all target objects whose sign encodes safety.

2. DeepReach and neural PDE-based reachability solvers

DeepReach established the basic neural HJR template: approximate the value function by a neural network $V$5, compute $V$6 and $V$7 by automatic differentiation, and train without supervision from precomputed value-function grids (Bansal et al., 2020). In the original formulation, $V$8 is parameterized by a fully connected network with sinusoidal activations; the later convergence analysis writes the architecture as

$V$9

with $\nabla_x V$0 (Hofgard, 2024).

The basic DeepReach loss combines a terminal-condition term and a PDE-residual term. In the HJI setting,

$\nabla_x V$1

and

$\nabla_x V$2

DeepReach trains with a terminal-condition pretraining phase, then a backward-time curriculum in which time is progressively marched from $\nabla_x V$3 toward $\nabla_x V$4 (Bansal et al., 2020).

The convergence study of DeepReach modifies this design by replacing $\nabla_x V$5 residuals with a supremum-norm loss,

$\nabla_x V$6

$\nabla_x V$7

where $\nabla_x V$8 (Hofgard, 2024). The paper emphasizes that $\nabla_x V$9 is structurally aligned with reachability because reachability is a worst-case, boundary-sensitive property: small localized errors can move the $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$0 boundary substantially even when an $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$1 loss is small.

In practice, these methods approximate the sup norm by sampling. The convergence paper uses large batches, maximum-over-batch residuals, and a DeepReach-style training schedule consisting of terminal-condition pretraining followed by backward-time training with $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$2 sampled states per step, with time and state variables rescaled to $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$3 (Hofgard, 2024). The learned value function is operationally useful because the optimal safety control is obtained from the Hamiltonian: $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$4 so neural HJR learns both a safety certificate and a feedback law (Bansal et al., 2020).

3. Alternative learning paradigms and structural priors

Although DeepReach is the reference neural PDE solver, neural HJR has diversified along several methodological directions. One line improves the function class and loss design. NeHMO introduces residual learning with exact boundary imposition,

$$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$5

so that the network learns only the deviation from the signed distance and the terminal condition becomes $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$6. The same work exploits symmetry: if the boundary function and Hamiltonian are invariant under a differentiable bijection $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$7, then $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$8, allowing training on only half of the state space and inference by symmetry completion (Chen et al., 18 Jul 2025).

A second line changes the geometry of the HJ problem itself. In manifold-constrained HJR, the state is restricted by equality constraints $$\dot{x} = f(x,u,d),\quad x\in\mathbb{R}^n,\ u\in\mathcal U,\ d\in\mathcal D,$$9, defining a manifold $L=\{x:\ell(x)\le 0\}$0. Dynamic programming on $L=\{x:\ell(x)\le 0\}$1 yields a constrained Hamiltonian

$L=\{x:\ell(x)\le 0\}$2

For velocity-controlled systems $L=\{x:\ell(x)\le 0\}$3, this reduces to

$L=\{x:\ell(x)\le 0\}$4

with projection

$L=\{x:\ell(x)\le 0\}$5

This gives a closed-form manifold-aware Hamiltonian that can be inserted directly into a DeepReach-style residual loss (Chen et al., 5 Nov 2025).

A third line changes the learning target. HJRNO uses a Fourier Neural Operator to learn an operator mapping terminal value functions to time-$L=\{x:\ell(x)\le 0\}$6 value functions across obstacle families and parameter settings, rather than learning a single value function instance. In the dynamic Dubins-car experiments, it learns from $L=\{x:\ell(x)\le 0\}$7 pairs generated by a classical HJ solver, generalizes across random obstacle shapes and hyperparameters, and therefore functions as an amortized surrogate for repeated reachability solves (Li et al., 28 Apr 2025).

A fourth line augments neural HJR with external optimal-control supervision. MPC-guided reachability learning uses a hybrid loss that combines PDE residuals with approximate value labels produced by a sampling-based MPC solver at collocation points, then iteratively refines the MPC labels using the current learned value function as terminal cost (Feng et al., 4 May 2025). A related idea appears in conservative linear envelopes for nonlinear reachability, where the discrepancy between a nonlinear system and a linear model is converted into an adversarial bounded artificial disturbance; the resulting generalized Hopf-formula solution yields conservative reachable sets and control laws and can serve as supervision or uncertainty-aware structure for learning-based HJR (Sharpless et al., 2024).

A fifth line replaces PDE residuals by Bellman updates. In reachability-aware shared control, the state-value and state-action reachability functions $L=\{x:\ell(x)\le 0\}$8 and $L=\{x:\ell(x)\le 0\}$9 are learned offline from Bellman-type equations and then used to define the Collision Avoidance Reachable Set (CARS), a hard safety constraint for shared-control RL (Zhao et al., 14 Feb 2025). SAGE learns an HJ-style safety Q-function directly from ego-vision by a discounted Bellman update,

$$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$0

and uses it as a least-restrictive safety shield around a performance policy (Chen et al., 2021). In visual navigation, HJ reachability is used as an offline expert that generates robust supervision for a waypoint-predicting CNN by modeling waypoint-prediction error as disturbance in the dynamics (Li et al., 2019). These variants suggest that neural HJR is not a single algorithmic family but a common value-learning viewpoint centered on HJ semantics.

4. Convergence theory, formal verification, and probabilistic certification

The most explicit PDE-theoretic guarantee presently available for neural HJR is the convergence theorem for the supremum-norm DeepReach loss. On a compact domain $$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$1, under compact $$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$2, bounded dynamics, and the assumption that the HJI equation admits a unique classical solution $$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$3 that is also the unique viscosity solution, any sequence of network parameters $$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$4 with $$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$5 satisfies

$$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$6

The proof treats each network as a solution of a perturbed HJI equation, rewrites the HJI variational inequality as a proper fully nonlinear PDE, applies relaxed-limit stability for viscosity solutions, and concludes by the comparison principle. The central implication is method-agnostic: the guarantee depends on the loss going to zero, not on a particular optimizer (Hofgard, 2024).

A separate verification program addresses a different question: how to certify the zero-sublevel set of a learned value function even when exact PDE convergence is unavailable. In discrete-time safe control, a value network $$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$7 defines a feasible region $$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$8, and safety follows if two implications hold: $$J(x,t,u(\cdot),d(\cdot)) = \min_{\tau\in[t,T]} \ell\big(\xi_{x,t}^{u,d}(\tau)\big),$$9 “Scalable Synthesis of Formally Verified Neural Value Function for Hamilton-Jacobi Reachability Analysis” formulates these checks as MILPs over ReLU networks and proposes three scalability devices: boundary-guided backtracking (BGB) for efficient counterexample search, entering state regularization (ESR) for enlarging the certified feasible region, and activation pattern alignment (APA) for reducing verification time by decreasing activation-pattern complexity. The same work introduces the Cersyve-9 benchmark and reports verified neural value functions on all nine tasks (Yang et al., 2024).

A third guarantee type is probabilistic rather than deterministic. “From Global to Local: Hierarchical Probabilistic Verification for Reachability Learning” constructs a coarse safe set by scenario optimization and then expands the certified region online near its boundary by local refinement. The method provides probabilistic safety guarantees for both the global certified set and the locally refined set and uses a switching mechanism between a learned reachability policy and a model-based controller (Smith et al., 26 Mar 2026). This addresses the practical regime in which neural HJR is accurate enough to be useful but not exact enough to admit classical deterministic HJ guarantees.

5. Representative domains and empirical performance

DeepReach demonstrated that neural HJR can reproduce classical reachability solutions in low dimension and remain usable far beyond the range of grid methods. On the 3D Air3D benchmark, it achieved MSE $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$0 versus the Level Set Toolbox solution and a BRT volume error of $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$1; it was also demonstrated on a 9D multi-vehicle collision problem and a 10D narrow-passage reach-avoid problem (Bansal et al., 2020). The convergence paper’s collision-avoidance experiments further showed that fine-tuning a pretrained DeepReach model with a supremum-norm loss for only $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$2k epochs substantially reduced maximum absolute error and improved the BRT boundary, especially in the $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$3 head-on slice (Hofgard, 2024).

In decentralized robotics, NeHMO provided some of the clearest evidence that neural HJR can be operational in real time. For particle systems, it maintained zero collision rate at 16 and 32 agents with success rates of $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$4 and per-agent planning times of $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$5 ms and $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$6 ms, whereas the same optimization framework driven by a DeepReach-style value function degraded to $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$7 success with $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$8 collision rate at 16 agents and $$V(t,x)=\inf_{d(\cdot)}\sup_{u(\cdot)} J(x,t,u(\cdot),d(\cdot)).$$9 success with $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$0 collision rate at 32 agents (Chen et al., 18 Jul 2025). On a 12-DoF dual-UR5 task, NeHMO achieved success rate $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$1, collision rate $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$2, and planning time $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$3 ms, compared with $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$4 success and $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$5 collision rate for the Opt+DeepReach baseline (Chen et al., 18 Jul 2025). In manifold-constrained multi-arm planning, HaMMAR reported success rate $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$6, collision rate $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$7, and planning time $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$8 s on the doorway-crossing UR5 task, whereas the same trajectory optimizer without HJR achieved success rate $$\min\left\{\partial_t V(t,x)+H(t,x),\ \ell(x)-V(t,x)\right\}=0,\qquad V(T,x)=\ell(x),$$9 and collision rate $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$0 (Chen et al., 5 Nov 2025).

Operator-learning and hybrid-supervision variants emphasize amortization and scale. HJRNO reported average error $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$1 on $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$2 random obstacle scenarios and inference time $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$3 s after training on HJ-generated data (Li et al., 28 Apr 2025). MPC-guided reachability learning reported a verified safe volume of $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$4 on a 13D quadrotor and a recovered BRT volume of $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$5 on a 40D publisher-subscriber system, substantially improving over residual-only DeepReach baselines (Feng et al., 4 May 2025).

Neural HJR has also been integrated into perception, shared control, and learned safety filtering. Reachability-aware shared control precomputed a reachability distribution and CARS, then deployed an RL policy on a real vehicle platform with average computation time $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$6 ms per $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$7 ms cycle (Zhao et al., 14 Feb 2025). SAGE learned an HJ-style safety critic directly from ego-vision and reported state-of-the-art results on the Learn-to-Race benchmark according to the paper’s abstract (Chen et al., 2021). In visual navigation, HJ-generated supervision for waypoints achieved success rate $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$8 versus $$H(t,x)=\sup_{u\in\mathcal U}\inf_{d\in\mathcal D}\nabla_x V(t,x)^\top f(x,u,d).$$9 for heuristic waypoint supervision in cluttered indoor environments (Li et al., 2019). NeuroHJR, a PINN-based obstacle-avoidance framework, reported an average travel-time reduction of $\mathrm{BRT}(t)=\{x:V(t,x)\le 0\}.$0 and path efficiency improvement of $\mathrm{BRT}(t)=\{x:V(t,x)\le 0\}.$1 relative to a conventional HJR solver in its simulations (Halder et al., 1 Dec 2025).

6. Limitations, failure modes, and open questions

The strongest current convergence theorem is conditional. It assumes a unique classical solution $\mathrm{BRT}(t)=\{x:V(t,x)\le 0\}.$2, compactness, bounded dynamics, and the existence of a sequence of neural parameters driving the supremum-norm loss to zero (Hofgard, 2024). Many HJI problems only admit viscosity solutions that are not $\mathrm{BRT}(t)=\{x:V(t,x)\le 0\}.$3, and extending the theory to purely viscosity solutions without classical regularity remains open (Hofgard, 2024). Even when the theorem applies, finite-sample training only approximates the sup norm, and high-dimensional sampling can be unreliable (Hofgard, 2024).

More broadly, neural HJR still trades exactness for scale. DeepReach itself states that it does not remove the curse of dimensionality in general; rather, its empirical cost scales more with the complexity of the reachable tube than with a grid size, and arbitrarily complex value functions can still be hard to learn (Bansal et al., 2020). Multi-agent worst-case formulations are conservative and can produce longer paths or deadlock-like behavior when cooperative solutions exist (Chen et al., 18 Jul 2025). Manifold-constrained variants inherit additional execution issues, because discrete trajectory segments can drift off the manifold without high-frequency projection or low-level correction (Chen et al., 5 Nov 2025). PINN-style obstacle-avoidance approximations, such as NeuroHJR, likewise do not come with exact safety guarantees because the value function and policy are only approximate (Halder et al., 1 Dec 2025).

Open directions are correspondingly clear. The convergence paper identifies extension to non-classical viscosity solutions, quantitative error rates linking loss to $\mathrm{BRT}(t)=\{x:V(t,x)\le 0\}.$4, and adversarial or adaptive sampling strategies as immediate theoretical and algorithmic goals (Hofgard, 2024). NeHMO points to post-training certification by scenario optimization, conformal prediction, and Lipschitz-based certificates; less conservative game formulations; and extensions to uncertainty, partial observability, and more complex dynamics (Chen et al., 18 Jul 2025). Verification work suggests that hybrid pipelines—global certification, local online refinement, and switching between learned and model-based controllers—may be the most practical route to deployable neural HJR in safety-critical systems (Smith et al., 26 Mar 2026). A plausible implication is that the field is converging on a layered architecture: neural value-function learning for scalability, structural priors for sample efficiency, and verification or certification layers for safety credibility.