Hybrid Action Representations

• Hybrid action representations are frameworks that model action spaces by coupling discrete decisions with continuous parameters, thereby tackling combinatorial explosion and preserving semantic structure.
• Techniques like conditional VAEs and latent space mapping enable efficient policy learning, robust generalization, and tractable control in domains such as robotics and quantum circuit synthesis.
• These representations facilitate perception-action alignment and formal verification, offering innovative solutions in action recognition, planning, and cyber–physical system control.

Hybrid action representations refer to frameworks and methodologies for representing, learning, and exploiting action spaces in sequential decision-making, perception, and imitation tasks where actions possess both discrete and continuous attributes. Such hybrid action spaces are ubiquitous in robotics, task and motion planning, quantum circuit synthesis, control of cyber–physical systems, action recognition in videos, and more. The technical challenges in hybrid action representation lie in maintaining computational tractability, expressiveness, and semantic alignment between discrete choices and continuous parameters, while enabling efficient learning and robust generalization.

1. Formal Definitions and Motivation

Hybrid action spaces are action sets of the form

$$\mathcal{A} = \{ (a_d, x_c):\, a_d \in \mathcal{A}_D,\; x_c \in \mathcal{X}_C(a_d) \subseteq \mathbb{R}^m \}$$

where $\mathcal{A}_D$ is a finite discrete set and $\mathcal{X}_C(a_d)$ is a (possibly action-dependent) continuous domain. Typical examples include selecting a motion primitive (discrete) plus specifying a target coordinate (continuous), choosing a controller mode (discrete) together with control gains (continuous), or selecting a robot end-effector (discrete) and a motion velocity (continuous) (Li et al., 2021, Zhang et al., 9 Dec 2025).

The need for explicit hybrid representations arises because naïvely flattening the action space—by discretizing all continuous parameters or treating all actions as continuous-valued—rapidly leads to combinatorial explosion, numerical ill-conditioning, or loss of the underlying semantic structure. For instance, discretizing each continuous parameter to $N$ bins with $K$ discrete choices yields $K \times N^m$ possible actions, which is computationally infeasible when $K$ or $m$ is large (Li et al., 2021).

2. Latent Space and Decodable Hybrid Representations

A prevalent approach is to map each hybrid action $(a_d, x_c)$ onto a compact latent vector $z \in \mathbb{R}^d$ that supports both efficient policy learning and exact or approximate decoding back to the original hybrid action.

For example, HyAR represents the hybrid action using:

• A learnable discrete action embedding table $E_\zeta(a_d) \in \mathbb{R}^{d_1}$
• A conditional variational autoencoder (VAE) encoder $q_\phi(z_x | s, a_d, x_c)$ that encodes the continuous parameter, conditioned on the state $s$ and $E_\zeta(a_d)$
• Decoding is performed by nearest-neighbor lookup in $E_\zeta$ for the discrete action and by a conditional VAE decoder $f_\theta$ for the continuous parameter (Li et al., 2021)

The representation is trained to minimize a VAE reconstruction loss plus a regularization loss that enforces “semantic smoothness,” i.e., that neighbors in latent space yield similar transitions in state space, using an unsupervised dynamics-prediction auxiliary objective. This construction enables standard off-the-shelf continuous-control RL policies (e.g., TD3, DDPG) to operate on $z$, while guaranteeing a mapping to meaningful hybrid actions and stable learning dynamics.

A similar approach is adopted for quantum circuit search, where the hybrid action couples discrete gate choice with continuous parameter assignments and refinement (Niu et al., 7 Nov 2025).

3. Alignment and Abstraction: Mirror and Linguistic Representations

Hybrid action representations are also studied through the lens of representational alignment, inspired by biological systems such as mirror neurons. Representation alignment methods map observed (e.g., video) and executed (e.g., policy) actions into a shared latent space, enforcing correspondence via contrastive loss and maximizing mutual information between modalities (Zhu et al., 25 Sep 2025). This bidirectional alignment enables robust transfer between perception and control, leveraging the structural and functional couplings inherent to hybrid actions.

In robotic manipulation, hybrid representations can mediate between language, high-level motion abstraction, and fine-grained control. Recent work proposes a two-stage approach: first, an abstract, language-like “motion” token sequence captures direction and modality (without magnitude), making the representation scale-invariant and semantically aligned with language tokens; second, a sequence of discretized action-bin tokens encodes the fine-grained actuator commands (Zhang et al., 9 Dec 2025). This layered structure narrows the feature distance between language and low-level actions, enabling efficient transfer learning and generalization.

4. Hybrid Representations in Sequential Learning and Planning

Hybrid action spaces are central in reinforcement learning, planning, and control, with sophisticated techniques for policy and value-function representation:

• Hybrid Policy Parameterization: In multi-agent hybrid soft actor-critic (MAHSAC), the policy for each agent is factored into a discrete component $\pi^d(a^d|o)$ and a conditional continuous component $\pi^c(a^c|o, a^d)$, enabling joint entropy regularization and sample-efficient training via centralized critics (Hua et al., 2022, Hua et al., 2022).
• Factored Hybrid MDPs: Hybrid factored MDPs represent state and action as both discrete and continuous factors. The value function is approximated as a linear combination of basis functions over the hybrid state space, and optimization is performed via hybrid approximate linear programming (HALP), where hybrid constraints are handled by cutting-plane and sampling methods (Guestrin et al., 2011).
• Active Inference and Option Discovery: Hybrid active inference models utilize a hierarchical structure, with a high-level discrete planner that reasons over dynamically learned discrete modes (via rSLDS) and low-level continuous controllers specialized to each mode. Discrete transitions instantiate temporally-abstract options, and the architecture facilitates information-theoretic exploration and efficient planning (Collis et al., 2 Sep 2024, Priorelli et al., 1 Feb 2024).

Such structures are crucial for system identification, exploration, and temporally-abstracted planning in domains with continuous dynamics and discrete event structure.

5. Hybrid Architectures in Perception and Imitation

Hybrid action representations extend beyond policy learning to action perception, recognition, and imitation:

• Action Assessment in Video: The ACTION-NET hybrid architecture uses dual streams: a dynamic (I3D) stream for temporal motion cues and a static (ResNet) stream for posture analysis, fused via a context-aware graph convolutional attention module. This hybrid captures both movement and execution quality, which is essential in sports and skill assessment (Zeng et al., 2020).
• Hybrid Temporal Abstraction: In imitation learning (HYDRA), a hybrid action space combines high-level, sparse waypoints (for temporal abstraction and planning) with low-level, dense velocity commands (for dexterity). An explicit mode variable selects between the two; offline action relabelling improves action consistency during temporal abstraction phases, mitigating distribution shift in behavioral cloning (Belkhale et al., 2023).
• Hybrid Feature Pipelines: In video action recognition, hybrid architectures often combine unsupervised feature-extraction (e.g., hand-crafted dense trajectories, Fisher Vectors) with deep supervised classifiers. This strategy captures both fine-grained spatio-temporal features and supports powerful function approximation, achieving state-of-the-art performance on limited data (Souza et al., 2016).

6. Representation Languages and Formal Synthesis

Hybrid action representations also underpin formal modeling of hybrid automata and cyber–physical systems:

• Action Languages Modulo Theories: Action languages such as ${\cal C}+$ can encode both discrete events and continuous evolution (ODEs), representing hybrid automata with arbitrary mode structure, guard expressions, reset maps, and continuous flows. These high-level models compile to satisfiability modulo theories (SMT), and, when extended with ODE semantics, to SMT(ODE), enabling formal analysis, reachability computation, and verification using solvers like dReal (Lee et al., 2017).
• This formalism provides declarative, modular, and elaboration-tolerant representations for complex dynamic systems with coupled discrete and continuous transitions.

7. Empirical Results and Impact

Hybrid action representations yield substantial empirical advantages across domains:

Domain / Task	Hybrid Representation	Key Empirical Gain
Hybrid-action RL (HyAR, MAHSAC)	Latent/structured hybrid	Scalability, stability, superior sample efficiency, robust to large action spaces (Li et al., 2021, Hua et al., 2022)
Perception-Action Alignment	Mirror latent, contrastive	Enhanced generalization, higher action recognition and execution rates (Zhu et al., 25 Sep 2025)
Multimodal manipulation	Linguistic+token-action	Transferability, cross-platform generalization, mitigated distribution shift (Zhang et al., 9 Dec 2025)
Classical control, MDP planning	Hybrid factored / ALP	Tractability, closed-form guarantees, escape from discretization bottleneck (Guestrin et al., 2011)
Video recognition	Hand-crafted + deep hybrid	SOTA accuracy, data efficiency, strong transferability (Souza et al., 2016)

Ablation studies across these works typically show that (i) omitting hybrid-specific components (e.g., conditional VAEs, hierarchical structure, or mode selectors) results in marked degradation; (ii) advances in smooth latent alignment, representation relabelling, and context-aware fusion each notably contribute to final performance.

• (Li et al., 2021) HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation
• (Zhang et al., 9 Dec 2025) Bridging Scale Discrepancies in Robotic Control via Language-Based Action Representations
• (Zhu et al., 25 Sep 2025) Embodied Representation Alignment with Mirror Neurons
• (Hua et al., 2022, Hua et al., 2022) Deep Multi-Agent RL with Hybrid Action Spaces
• (Guestrin et al., 2011) Solving Factored MDPs with Hybrid State and Action Variables
• (Lee et al., 2017) Representing Hybrid Automata by Action Language Modulo Theories
• (Belkhale et al., 2023) HYDRA: Hybrid Robot Actions for Imitation Learning
• (Souza et al., 2016) Sympathy for the Details: Dense Trajectories and Hybrid Classification Architectures for Action Recognition
• (Zeng et al., 2020) Hybrid Dynamic-static Context-aware Attention Network for Action Assessment in Long Videos
• (Niu et al., 7 Nov 2025) Hybrid action Reinforcement Learning for quantum architecture search
• (Collis et al., 2 Sep 2024) Learning in Hybrid Active Inference Models
• (Priorelli et al., 1 Feb 2024) Deep hybrid models: infer and plan in a dynamic world

Hybrid action representations now form a foundational component of modern approaches to learning, perception, control, and reasoning in complex dynamic environments with intertwined discrete and continuous structure.