Latent Action Policies from Observation

• LAPO is a framework that learns latent action spaces from observation sequences without explicit action labels, enabling imitation and reinforcement learning.
• It employs inverse and forward dynamics modeling with information bottlenecks to ensure deterministic, disentangled, and informative latent codes.
• LAPO methods improve data and sample efficiency in robotics and cross-domain tasks while mitigating distractor effects through minimal supervision.

Latent Action Policies from Observation (LAPO) correspond to a family of methodologies and theoretical insights that enable the learning of action policies directly from observation sequences—typically visual or state trajectories—without using explicit action labels. The LAPO framework centers on the inference of a latent action space that captures causal intervention in the environment, thereby facilitating imitation learning, policy transfer, and reinforcement learning from data where action labels are absent or costly to procure. The field encompasses both practical algorithms and recent theoretical analysis on the conditions under which these latent representations are identifiable and useful.

1. Foundations and Desiderata of Latent Action Representations

The foundational theoretical analysis of LAPO formalizes three desiderata for effective latent action representations (Lachapelle, 1 Oct 2025):

• Determinism: The encoder should assign a unique latent code to each state–transition pair that corresponds to a specific action, removing stochasticity in the mapping and ensuring reproducibility.
• Disentanglement: The latent action should depend only on the true action, not on the particular starting state. This ensures that the semantics of the latent code are invariant to the agent's current configuration, aligning each latent with a ground-truth action independent of incidental context.
• Informativeness: The mapping from expert actions to latent codes should be injective. No two distinct expert actions may map to the same latent code; this is a requirement for later recovering (or decoding) the true action space from the latent actions with minimal supervision.

These properties ensure that, after learning on a large corpus of unlabeled state–transition data, each transition is labeled with a latent code that both captures all information about the expert action and can be deterministically.

2. Methodological Frameworks

Several LAPO approaches share a structural paradigm:

• Latent Inverse and Forward Dynamics Modeling: The core is to train an inverse dynamics model (IDM) that infers a latent action from consecutive observations (e.g., oₜ, oₜ₊₁) and a forward dynamics model (FDM) that predicts the next observation given the current observation and a latent action. The coupling of these models ensures that the latent action encodes information necessary for predicting realistic transitions (Schmidt et al., 2023, Ye et al., 15 Oct 2024).
• Information Bottleneck: To prevent the IDM from trivially copying future information, constraints (such as vector quantization bottlenecks or limiting latent capacity) are imposed so that the latent action can capture at most the information necessary to explain the transition.
• Decoding Step: Once the latent actions have been inferred and a latent policy π(oₜ) → zₜ has been fitted (via behavior cloning or a similar imitation objective), a small, supervised dataset suffices to train a simple decoder that maps zₜ (and possibly oₜ) to real actions, or a further phase of online RL is used to fine-tune the latent model to the true action space (Schmidt et al., 2023, Ye et al., 15 Oct 2024, Liang et al., 8 May 2025).

A representative loss for the unsupervised phase is

$L = \lVert \hat{o}_{t+1} - o_{t+1} \rVert^2$

with $\hat{o}_{t+1}$ produced from FDM and latent zₜ, and zₜ itself produced as an information-bottlenecked function of $(o_t, o_{t+1})$. Decoding or action-alignment loss is typically a supervised MSE or cross-entropy via a learned mapping from latent space to action space.

3. Theoretical Guarantees and Identifiability

A central question is under what conditions the latent action space is identifiable from data. The theoretical analysis (Lachapelle, 1 Oct 2025) demonstrates:

• Under an entropy-regularized objective,

$$\min_{f̂, q̂} \mathbb{E}_{x,x'}\left[ \mathbb{E}_{\hat{a}\sim q̂(\cdot|x,x')}\lVert x' - f̂(x,\hat{a}) \rVert^2 + \beta H(q̂(\cdot|x, x')) \right],$$

where $H(\cdot)$ is entropy, with sufficiently large β and infinite data, the solution yields deterministic, injective, and disentangled latent action codes that uniquely label expert actions according to the desiderata above. Discrete latent spaces naturally encourage these properties and have proven effective in practice for mapping expert action spaces, as in Genie, LAPA, and Schmidt et al. 2024.

Violations of identifiability occur when:

• The hypothesis space is insufficiently restricted (e.g., IDM outputs the next observation itself as latent code).
• The underlying expert policy is deterministic, collapsing all transitions to a single code unless forced otherwise.

4. Statistical and Practical Benefits

When the latent action representations satisfy the desiderata, the statistical benefits are substantial:

• Data Efficiency: Unlabeled video or state–transition data can be automatically annotated with high-quality latent actions. This allows the agent to imitate or learn from large corpora with minimal labeled action data—often requiring only a small alignment set.
• Sample Efficiency: Since the latent-to-real-action decoder is simple (often a linear or shallow model), only a handful of expert-labeled examples are needed to ground the latent space in the executable action space.
• Regularization: The bottlenecked latent representation reduces overfitting to spurious correlations and mimics the effect of a denoiser, enhancing the generalization of the learned policy.

In both simulation and real-world tasks—including classic control, procedurally generated environments, and robotic manipulation—LAPO-based methods have demonstrated superior data efficiency, often achieving expert-level behavior with a small supervised alignment phase after unsupervised latent action learning (Schmidt et al., 2023, Ye et al., 15 Oct 2024, Liang et al., 8 May 2025).

5. Challenges: Distractors and Robustness

Practical deployment of LAPO faces several challenges, especially in observational data replete with distractors:

• Action-Correlated Distractors: In naturalistic videos, changes in observation may arise from factors not controlled by the agent (e.g., moving backgrounds, camera shake). Standard LAPO can overfit to such features, reducing downstream performance (Nikulin et al., 1 Feb 2025, Klepach et al., 13 Feb 2025).
• Remedies: Methods addressing distractors include a) moving to multi-step IDM (to encourage persistent, task-relevant changes), b) increasing latent capacity, c) replacing pixel-level losses with latent consistency losses, d) employing data augmentations, and—critically—e) incorporating minimal supervision (e.g., using as little as 2.5% of labeled actions during latent training) (Nikulin et al., 1 Feb 2025).
• Object-Centric Representation: Integrating self-supervised object decomposition into LAPO (object-centric pretraining) strongly mitigates distractor effects, with downstream task performance improved by up to 2.6× and action recovery by 50%, as shown in DCS and DMW environments (Klepach et al., 13 Feb 2025).

6. Applications and Cross-Domain Transfer

LAPO has found wide application in:

• Robotics and Manipulation: Learning from vast, unlabeled video of both humans and robots, including cross-embodiment transfer using semantically-aligned latent spaces for different robot morphologies (Bauer et al., 17 Jun 2025, Ye et al., 15 Oct 2024).
• Language-Vision-Action Models: Pretraining generalist agents that map language instructions and observations to behaviors using latent action quantization, outperforming even action-labeled VLA models on complex manipulations and reducing pretraining cost by 30–40× (Ye et al., 15 Oct 2024).
• Reinforcement Learning: Offline policy learning from image-based or heterogeneous sensory data, with policy optimization and planning conducted entirely in the learned latent action space, incorporating distributional constraints and uncertainty regularization for robust performance (Alles et al., 7 Nov 2024).
• Imitation Learning: Enabling behavior cloning from videos and observations, even when expert actions are unavailable, and allowing rapid adaptation/fine-tuning with minimal labeled intervention (Schmidt et al., 2023, Liang et al., 8 May 2025).

7. Impact, Open Problems, and Future Directions

LAPO methodologies unlock new avenues for scaling policy and world model pretraining to web-scale observational data, sharply reducing reliance on action-annotated demonstrations. The key challenges ahead involve:

• Scaling to diverse, noisy, real-world data where action-correlated distractors are prevalent.
• Automating object selection in object-centric frameworks and improving latent-to-action alignment mechanisms across morphology and domain gaps.
• Deepening theoretical characterizations of identifiability and regularization choices, possibly moving beyond discrete codes to more flexible continuous or compositional action representations.

Recent empirical and theoretical results suggest that appropriately regularized, discrete latent spaces, combined with minimal supervision and structured representations, will remain foundational for robust LAPO systems in embodied AI (Lachapelle, 1 Oct 2025, Klepach et al., 13 Feb 2025, Liang et al., 8 May 2025).

---

Property	Role in LAPO Identifiability	Effect in Practice
Determinism	Unique code per transition	Stable, reproducible proxy action labeling
Disentanglement	Latent independent of state	Decoding is simple, improves generalization
Informativeness	Injective mapping	Full recoverability of expert actions

These properties, now formalized and established under entropy-regularized objectives, underpin the efficiency and transferability of latent action policies from observation.