Latent-Space Safe Control

• Latent-space safe control is defined by performing safety analysis, controller synthesis, and runtime filtering on compact, learned representations instead of full physical states.
• It leverages diverse latent objects—such as world-model latent states, belief distributions, zonotopes, and latent action codes—to implement barrier functions, reachability, and robust MPC.
• Applications span autonomous driving, robotics, and process control, with challenges including uncertainty management, encoder distortions, and transferring latent guarantees to real-world systems.

Searching arXiv for papers on latent-space safe control, latent safety filters, latent CBFs, and latent-space reachability. I’ll look up the most relevant arXiv entries to ground the article in current literature. Across recent work, latent-space safe control denotes control design in which safety analysis, controller synthesis, or runtime filtering is performed on a compact latent representation—such as a latent state, latent belief, or latent action code—rather than on the full physical state or raw observations. The motivation is consistent across the literature: verification or planning in the original space may be computationally prohibitive, partially observed, or difficult to model, whereas a learned latent space can support barrier certificates, reachability analysis, robust model predictive control, or calibrated safety filtering at tractable dimension while still aiming to preserve physical safety properties (Lutkus et al., 29 May 2025, Anand et al., 18 Jul 2025, Nakamura et al., 2 Feb 2025, Nath et al., 14 Jun 2026). The resulting research program includes continuous world models, belief-space formulations, set-valued latent verification, runtime-adaptable safety constraints, and adjacent formulations based on discrete latent states, perception-feature novelty regions, and latent action manifolds (Wu et al., 7 Apr 2026, Agrawal et al., 23 Sep 2025, Han et al., 2019, Pulsipher et al., 2022, Koirala et al., 2024).

1. From latent risks to latent safety representations

Earlier related formulations used latent or latent-like internal representations without yet centering continuous neural world models. In autonomous driving, an evolving finite state machine (e-FSM) was used to discover discrete latent situation states online, including a “Dead-End” state with latent-risk of collision, and to learn action-conditioned transition matrices over those states (Han et al., 2019). In process control, SAFE-OCC used feature maps from convolutional blocks of a CNN sensor as the feature space for one-class novelty detection, thereby defining a safe region in a sensor-induced latent space rather than in the original image space (Pulsipher et al., 2022).

More recent work moved to continuous learned representations intended for direct controller synthesis. A formal study of low-dimensional latent representations for verifiable control synthesis was initiated in “Latent Representations for Control Design with Provable Stability and Safety Guarantees” (Lutkus et al., 29 May 2025). Its stated objective is to enable Lyapunov or barrier-function methods that might otherwise be computationally prohibitive on the full state, using dynamics-aware approximate conjugacy conditions and transfer of latent stability and invariance guarantees back to the original system (Lutkus et al., 29 May 2025). This suggests a shift from latent representations as descriptive surrogates toward latent representations as objects on which verification itself is performed.

A separate but related strand treats “latent risks” semantically rather than geometrically. “Context-aware LLM-based Safe Control Against Latent Risks” models hidden or unspecified risk variables through LLM-based reasoning, then translates them into subtask-level reward and safety constraints for MPC (Deng et al., 2024). Although this is not a world-model latent space in the usual sense, it broadens the meaning of latent-safe control to include semantic latent variables that shape safety constraints.

2. Latent variables used by safe controllers

The literature now contains several distinct latent objects. In visuomotor world-model settings, the central latent is a compact state $z_t$ inferred from images and, in some cases, proprioception. In partially observed systems, the latent object is often a belief state $z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$ together with predictive uncertainty, rather than a deterministic embedding (Nuchkrua et al., 31 Jan 2026). In risk-sensitive belief-space MPC, the relevant latent object can instead be a belief distribution $b_t(\theta)$ over an unobserved parameter $\theta$ that affects dynamics, costs, and safety margins (Enwerem et al., 4 Apr 2026). In set-based verification, the state is lifted from a point to a zonotope $\mathcal{X} = \langle x_c, G_x \rangle$, propagated through an encoder $E$, and evaluated as a latent zonotope (Wu et al., 7 Apr 2026). Offline safe RL introduces yet another variant: a latent variable $z$ in a CVAE that parameterizes safe actions rather than latent dynamics (Koirala et al., 2024).

These latent objects support different control architectures.

Family	Representative papers	Latent object
World-model latent state	(Anand et al., 18 Jul 2025, Nakamura et al., 2 Feb 2025, Nath et al., 14 Jun 2026)	Action-conditioned latent state for prediction and control
Latent belief or latent parameter belief	(Nuchkrua et al., 31 Jan 2026, Enwerem et al., 4 Apr 2026)	Belief over unobserved state or parameter
Set- or constraint-parameterized latent state	(Wu et al., 7 Apr 2026, Agrawal et al., 23 Sep 2025)	Latent zonotope or image-conditioned constraint embedding
Latent action constraint	(Koirala et al., 2024)	Latent code generating safe actions

A common misconception is that latent-space safe control is tied to one representation family. The current literature instead shows at least four: latent states for world-model prediction, latent beliefs for partial observability, latent sets for uncertainty-aware verification, and latent action codes for policy restriction.

3. Safety formalisms in latent space

The dominant safety formalisms in latent space are control barrier certificates, Hamilton–Jacobi-style reachability, and MPC-based safe filtering. In “Safety Certification in the Latent space using Control Barrier Functions and World Models” (Anand et al., 18 Jul 2025), the discrete-time safety condition is written as

$B(f(x,u)) \le B(x),$

but the barrier $B_\theta$ and controller $\pi_\theta$ are learned in the latent space of a world model driven by visual observations. The method combines a supervised separation loss on labeled safe and unsafe latent samples, a Lie-like temporal regularizer, and an overview loss

$z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$0

which penalizes increases of the barrier under latent dynamics (Anand et al., 18 Jul 2025). The paper explicitly states that the framework encourages safer behaviour via the learned barrier but does not provide formal safety guarantees (Anand et al., 18 Jul 2025).

Reachability-based methods replace barrier monotonicity by a latent Bellman equation. “Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis” defines a learned failure classifier $z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$1 and a latent safety value

$z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$2

or a discounted version for RL-based approximation, yielding a Latent Safety Filter that overrides a base controller when the latent value of its imagined next state falls below a threshold (Nakamura et al., 2 Feb 2025). This formulation preserves the classical HJ interpretation of “doomed to fail” states, but its guarantees are approximate because the world model, the classifier, and the value function are all learned (Nakamura et al., 2 Feb 2025).

Later work identified limitations of least-restrictive switching and proposed smooth latent-space filtering. “How to Train Your Latent Control Barrier Function” shows that classifier-style margin functions produce saturated value functions and proves that the value function’s Lipschitz constant scales linearly with the margin function’s Lipschitz constant (Nakamura et al., 23 Nov 2025). It therefore regularizes the latent margin with gradient penalties and trains value functions on data from both nominal and safety policies, then enforces the discrete-time latent CBF inequality

$z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$3

to obtain smoother interventions (Nakamura et al., 23 Nov 2025).

Constraint parameterization extends this reachability formalism to runtime-adaptable safety specifications. “AnySafe” conditions the failure margin on an encoded constraint image $z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$4, using

$z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$5

and trains a universal value function $z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$6 in world-model imagination so that safety constraints can be changed at runtime without retraining (Agrawal et al., 23 Sep 2025).

4. Uncertainty, partial observability, and adaptive latent models

A central difficulty is that point estimates in latent space rarely capture the uncertainty structure relevant to safety. “Cognitive-Flexible Control via Latent Model Reorganization with Predictive Safety Guarantees” considers partially observable stochastic cyber-physical systems with latent beliefs $z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$7, surprise-driven model updates, and Bayesian MPC in latent space (Nuchkrua et al., 31 Jan 2026). The paper introduces a Cognitive Flexibility Index, bounded posterior drift, adaptive constraint tightening, recursive feasibility, and ISS of the closed-loop belief dynamics under incremental adaptation (Nuchkrua et al., 31 Jan 2026). In its reported abrupt-shift scenario, CF–DeepSSSM attains SafetyRate $z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$8 and ComfortCost $z_t \sim q_{\phi_t}(z_t \mid \mathcal{H}_t)$9, while a nominal fixed-model MPC violates safety and a robust fixed-tightening MPC is more conservative (Nuchkrua et al., 31 Jan 2026).

Belief-space optimization under latent parameter uncertainty takes a different route. “Risk-Constrained Belief-Space Optimization for Safe Control under Latent Uncertainty” maintains a particle belief $b_t(\theta)$0 over an unobserved parameter $b_t(\theta)$1, defines a trajectory safety margin $b_t(\theta)$2, and enforces

$b_t(\theta)$3

inside a risk-sensitive belief-space MPPI controller (Enwerem et al., 4 Apr 2026). The paper proves that this CVaR constraint implies $b_t(\theta)$4, recovers the risk-neutral optimum as the risk weight tends to zero, and extends the per-horizon safety guarantee to repeated MPC solves through a union bound (Enwerem et al., 4 Apr 2026).

Pointwise latent certification can also fail under bounded state-estimation error. “From Points to Sets: Set-Based Safety Verification in the Latent Space” propagates a state zonotope through the encoder, evaluates linear latent barrier heads on the resulting latent zonotope, and enforces safety on the worst case over the entire set (Wu et al., 7 Apr 2026). On a 16-dimensional quadrotor suspended-load gate passage task, set-valued evaluation achieves $b_t(\theta)$5 collision-free passages, compared to $b_t(\theta)$6 for point-based evaluation and $b_t(\theta)$7 for a fixed-margin baseline; it also reports blind spots where point evaluation falsely certifies safety (Wu et al., 7 Apr 2026). A crucial empirical result is that the safety gap between point and set evaluation varies up to $b_t(\theta)$8 across certificate heads, so no single fixed margin suffices (Wu et al., 7 Apr 2026).

Partial observability can invalidate the latent representation itself. “What You Don’t Know Can Hurt You” studies temperature-based failures and shows that RGB-only world models can induce myopic safety behavior, such as avoiding the observation of failure rather than preventing failure itself (Kim et al., 7 Oct 2025). The paper proposes a mutual-information-based diagnostic for whether observations capture safety-relevant features and a multimodal-supervised training strategy that uses additional sensing during training but not at deployment (Kim et al., 7 Oct 2025). This directly challenges the assumption, common in latent safety filters, that safety-critical features are observable in the learned latent state.

5. Guarantee transfer, conformal calibration, and probabilistic safety

Guarantees in latent-space safe control vary from abstract transfer results to empirical proxy guarantees. The most explicit transfer program appears in the abstract of (Lutkus et al., 29 May 2025): dynamics-aware approximate conjugacy conditions formalize the reconstruction error needed for systems analysis, and latent Lyapunov or barrier certificates can be transferred back to the original system to recover stability and invariance guarantees (Lutkus et al., 29 May 2025). Set-based latent verification extends that logic to uncertainty sets: if a control input satisfies the latent CBF condition on the whole latent zonotope, the corresponding original-space CBF inequality holds for all concrete states in the uncertainty set up to conjugacy error terms (Wu et al., 7 Apr 2026).

A second route replaces exact transfer by calibrated probabilistic bounds. “Pixels to Proofs” trains an action-conditioned joint-embedding world model with compact Markovian latent states, learns a latent constraint checker, and uses conformal prediction to calibrate both latent dynamics error sets and latent constraint classification (Nath et al., 14 Jun 2026). These calibrated sets are embedded into a GPU-accelerated SLS robust MPC scheme. The central theorem states that, under exchangeability and robust-correctness assumptions, the true latent trajectory and controls lie inside the SLS tubes with probability at least $b_t(\theta)$9 (Nath et al., 14 Jun 2026). In simulation, the resulting controller improves both goal-reaching performance and safety over latent world-model and safe-planning baselines, and on several tasks it attains higher robust success than nominal or shield-based alternatives (Nath et al., 14 Jun 2026).

Conformal methods also appear in safety filtering rather than robust MPC. “Safe Control using Learned Safety Filters and Adaptive Conformal Inference” combines a learned HJ-style safety filter with adaptive conformal inference, dynamically adjusting its switching threshold based on observed over-optimism of the learned safety values (Huriot et al., 20 Apr 2026). The resulting guarantee is explicitly soft: the rate of incorrectly quantifying uncertainty in the predicted safety of the nominal policy is asymptotically upper bounded by a user-defined parameter (Huriot et al., 20 Apr 2026). “AnySafe” uses class-conditioned conformal calibration differently, by calibrating the similarity threshold $\theta$0 that defines whether a latent state is too close to a constraint image, thereby controlling how closely the system may approach a user-specified unsafe configuration (Agrawal et al., 23 Sep 2025).

These results clarify an important distinction. Some papers aim to transfer latent certificates to physical safety statements; others certify only latent tube containment, latent classifier validity, or asymptotic error rates of learned filters. The strength of the guarantee therefore depends on where the approximation error is accounted for.

6. Applications, limitations, and adjacent directions

The application domains already span classical benchmarks and hard-to-model visuomotor tasks. Reported case studies include cartpole stabilization and two-vehicle collision avoidance (Lutkus et al., 29 May 2025), inverted pendulum and Dubins car from visual observations (Anand et al., 18 Jul 2025), bag spilling and toppling cluttered objects with a Franka Research 3 manipulator (Nakamura et al., 2 Feb 2025), a dexterous stowing task with latent slot-pose uncertainty (Enwerem et al., 4 Apr 2026), quadrotor gate passage with a suspended load (Wu et al., 7 Apr 2026), wax overheating with a Franka Research 3 manipulator (Kim et al., 7 Oct 2025), car-following with latent Dead-End states (Han et al., 2019), and process control with CNN sensors under visual novelty (Pulsipher et al., 2022).

Several limitations recur. First, many methods rely on the latent space preserving safety-relevant structure. When that fails, latent filters can become myopic or miscalibrated (Kim et al., 7 Oct 2025). Second, formal guarantees are often incomplete because encoder distortion, world-model error, classifier error, and policy approximation compound. This is stated explicitly in the latent CBC framework and in latent reachability approaches that provide empirical rather than hard guarantees (Anand et al., 18 Jul 2025, Nakamura et al., 2 Feb 2025). Third, fixed margins are often inadequate: set-based latent verification shows that uncertainty gaps are anisotropic and time-varying (Wu et al., 7 Apr 2026). Fourth, computational cost remains substantial in robust or belief-space methods, such as Bayesian MPC with online belief propagation or MPPI over latent parameter particles (Nuchkrua et al., 31 Jan 2026, Enwerem et al., 4 Apr 2026).

The topic also extends beyond latent state dynamics. In safe offline RL, “Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning” learns a CVAE whose latent variable $\theta$1 parameterizes a conservatively safe action manifold, then optimizes reward by training an encoder inside that latent constraint space (Koirala et al., 2024). In high-frequency control, latent action chunks and Reuse-then-Refine improve smoothness, reduce jerk, and reduce exceed counts, which is safety-relevant even though formal safety certificates are not provided (Wang et al., 24 May 2026). This suggests that latent-space safe control is no longer confined to latent state estimation; it also includes latent action generation and latent policy restriction.

Taken together, the literature establishes latent-space safe control as a family of methods rather than a single formalism. Its unifying idea is to move safety reasoning into a learned low-dimensional representation where barrier functions, reachability operators, robust MPC, or calibrated filters become tractable. Its central unresolved question is not whether safety can be expressed in latent space, but under what representation, uncertainty, and calibration assumptions those latent safety claims remain faithful to the underlying physical system.