Unsupervised Action Discovery

Updated 19 March 2026

Unsupervised action discovery is a process that automatically detects, segments, and categorizes atomic and compositional actions from unlabeled sensory data.
It leverages information-theoretic objectives like mutual information maximization and intrinsic rewards to induce diverse and discriminable skill sets.
The approach enables practical applications in hierarchical reinforcement learning, video understanding, symbolic planning, and robotic manipulation, facilitating efficient task decomposition.

Unsupervised action discovery refers to the suite of computational frameworks for automatically detecting, segmenting, and characterizing the structure of actions—typically as atomic or compositional units—without access to any labeled supervision. This problem spans hierarchical reinforcement learning (HRL), video understanding, robotics, and symbolic planning. Across domains, the central goal is to autonomously acquire, from unstructured experiences or perceptual streams, a discrete or structured set of "actions," "skills," "options," or "interaction modes" that facilitate downstream problem solving, efficient exploration, or semantic understanding. The field synthesizes information-theoretic objectives (e.g., mutual information), intrinsic motivation, unsupervised segmentation, clustering, and sequence modeling.

1. Problem Formulations Across Domains

Unsupervised action discovery spans a spectrum of problem settings:

Hierarchical Reinforcement Learning (HRL): Discovering temporally extended options or subgoals from exploratory experience, with the aim of enabling abstraction, compositionality, and improved exploration in sparse-reward environments. This is typically framed as mining subgoals, skills, or options from raw experience without extrinsic task reward (Rafati et al., 2019, Hosseini et al., 2 Feb 2026, Yang et al., 2023, Wang et al., 2024, Atanassov et al., 2024, Park et al., 2023).
Video Understanding and Segmentation: Parsing raw video collections into temporally localized atomic actions or semantic steps by unsupervised means—partitioning sequences based on visual or multimodal cues, and clustering segments into a vocabulary (Piergiovanni et al., 2021, Spurio et al., 7 Aug 2025, Sener et al., 2016, Piergiovanni et al., 2021).
Symbolic Model Discovery: Inducing STRIPS/PDDL action schemas solely from unordered state transition traces, potentially with unknown action signatures, under constraints of generalization and parsimony (Suárez-Hernández et al., 2020).
Robotic Manipulation: Inferring discrete "interaction modes" (e.g., open, close, pull, push) underlying continuous control trajectories without labeled primitives, typically via unsupervised representation learning and clustering over collected task observations (Wang et al., 2024).

Although terminology varies (options, skills, subgoals, actions, primitives, interaction modes), the shared challenge is to recover a compositional structure of behavior or task decomposition that facilitates generalization and efficient learning without reliance on ground truth partitioning.

2. Information-Theoretic and Intrinsic Objectives for Skill Discovery

A central methodological axis is the use of information-theoretic objectives—primarily mutual information (MI)—and intrinsic motivation to drive unsupervised acquisition of skills:

Mutual Information Maximization: Many methods maximize $I(Z;S)$ , the MI between skill variable $Z$ (latent code) and state visitation $S$ , often with a variational posterior $q_\phi(z|s)$ providing a discriminative reward (Hosseini et al., 2 Feb 2026, Yang et al., 2023, Laskin et al., 2022, Kim et al., 2023, Atanassov et al., 2024, Park et al., 2023). This induces diverse, discriminable skills, each targeting a distinct subset of state space, but can collapse to static solutions if not regularized.
Distance- or Difficulty-Aware Maximization: Distance-maximizing skill discovery (DSD) encourages policies to move the environment along learned or task-driven "hard" directions, often subject to distance constraints or using learned state factorization (Hosseini et al., 2 Feb 2026, Atanassov et al., 2024, Park et al., 2023).
Curiosity and Empowerment: Intrinsic rewards based on empowerment (maximizing agent's influence over future states) or surprise (e.g., negative log-likelihood under a learned density model, -log q(s'|s)) promote discovery of controllable and underexplored behaviors (Binas et al., 2019, Hosseini et al., 2 Feb 2026). Controllability-aware methods adaptively shift the discovery pressure toward harder-to-control state transitions as previously easy transitions are mastered (Park et al., 2023).
Contrastive Skill Discovery: Recent approaches use contrastive learning to maximize skill-behavior discriminability via InfoNCE objectives, as in CIC (contrastive between transitions and skills) or BeCL (contrastive between pairs of behaviors under the same skill), and use embedding entropy as an intrinsic reward (see Table 1 below) (Laskin et al., 2022, Yang et al., 2023). This addresses the collapse-to-static/degenerate-skills problem in vanilla MI maximization.

Method/family	Core discovery objective	Salient regularizer/bonus
MI-based (DIAYN, SMM, DADS)	$\max I(Z; S)$	Discriminator: $q(z\|s)$
Distance-maximizing (DSD, LSD, METRA)	$\max (\phi(s')-\phi(s))^\top z$	Constraint: $\|\phi(s)-\phi(s')\| \le d(s, s')$
Controllability-aware (CSD)	DSD with $d(s,s') = -\log q(s'\|s)$	Hardness-driven reward
Contrastive (CIC, BeCL)	InfoNCE over $(s, s')$ or behaviors	Embedding entropy reward
Empowerment	$I($ option $;$ trajectory $)$	Option decoding accuracy

Empirical results, e.g. SUSD (Hosseini et al., 2 Feb 2026), show that factorization and curiosity-driven exploration (guiding the agent to target underexplored subspaces of the state factors) improve factor coverage, balance the discovery between environment components (entities, objects), and enable efficient downstream hierarchical RL.

3. Subgoal and Option Discovery in Hierarchical RL

In HRL contexts, unsupervised action discovery involves two elements: identifying promising subgoals and constructing temporally extended options or composite policies. Rafati & Noelle (Rafati et al., 2019) describe a two-level model-free HRL scheme in which:

Exploration is driven by intrinsic motivation for subgoal attainment, with experience memories mined periodically via anomaly detection (for rare, high-reward states) and K-means clustering (for typical but representative states).
Discovered subgoals (both anomalies and centroids) are turned into options with initiation sets, internal controller policies, and termination conditions.
The representation network $\phi(\cdot)$ is shared between state clustering and value functions, enabling unified learning without auxiliary models.
Agent coverage (e.g., in Four-Rooms and Montezuma's Revenge) is dramatically improved compared to standard random or non-hierarchical approaches, with subgoal-attainment rates exceeding 90%.

SUSD extends such ideas by factorizing the state and latent skill spaces, ensuring that all controllable factors are represented and that HRL controllers may later select and coordinate among multiple disentangled skills for complex tasks (Hosseini et al., 2 Feb 2026).

4. Unsupervised Action Segmentation in Video and Instructional Data

Video-based action discovery methods perform unsupervised temporal segmentation and clustering into action primitives. Canonical approaches include:

Sequential Stochastic Autoregressive Models: (Piergiovanni et al., 2021, Piergiovanni et al., 2021) use RNNs/grammars with Gumbel-Softmax sampling to probabilistically segment sequences. Self-labeling proceeds via stochastic rule sampling, followed by cost-based selection favoring action occurrence, length-consistency, and visual appearance agreement.
Generative Bayesian Nonparametrics: (Sener et al., 2016) models per-frame multimodal features (visual atoms, language atoms) with Beta-Process HMM priors, inferring both the number and semantics of steps. Segments are aligned across a collection by feature sharing; textual descriptors emerge via language modeling.
Mixed-Supervision Discovery: (Spurio et al., 7 Aug 2025) considers "action discovery" with only partial annotations (some actions labeled, others unknown), using annotated granularity to guide temporal segmentation (GGSM) and performing cluster assignment (UASA) on unlabelled segments using GMM and K-means over learned embeddings. This yields significant gains in F1 metrics for unknown actions compared to unsupervised or oracle baselines.

State-of-the-art models match or exceed weakly supervised methods on challenging datasets (NIV, 50Salads, Breakfast), demonstrating the effectiveness of constraints combined with self-labeling, cross-embedding losses, and hierarchical clustering.

5. Unsupervised Symbolic Action Model Induction

In classical planning, unsupervised action discovery targets the synthesis of STRIPS/PDDL action models from goal-oriented transition traces without intermediate state or action annotations (Suárez-Hernández et al., 2020). The approach is to:

Compile the schema discovery problem into a single parallel planning task, where edit-actions successively hypotheses headers, preconditions, and effects for $k$ candidate schemas with arity $r$ .
Use anomaly detection and symbolic constraints to systematically prune and generalize, including both statically determined predicates and variable-effect arities.
Leverage SAT-based planners to validate and select minimal-cost action models for all traces in parallel.
Empirical results on blocksworld, hanoi, and grid domains show that fully unsupervised discovered models generalize to all validation problems, with best solutions reached in modest compute times.

A key challenge is balancing parsimony with over-generalization, especially when only goal states (and not negative examples) are available.

6. Unsupervised Discovery of Robotic Action Modes

Recent robotic learning work has focused on discovering discrete interaction modes (primitives) underlying continuous control data without privileged labels (Wang et al., 2024). In ActAIM2:

A conditional GMM-VAE selector clusters task embeddings (derived as visual differences between initial and final object states), forming discrete interaction modes.
A behavior-cloning transformer predicts low-level action trajectories conditioned on both sensory input and the discovered discrete mode.
The pipeline operates entirely self-supervised, relying only on visually defined thresholds to label successful interaction attempts.
ActAIM2 achieves higher sample success rates in both seen and unseen objects/categories relative to unsupervised and weakly supervised baselines, with robust grounding in real-robot demonstrates.
Empirical ablations demonstrate mode separability is critically dependent on informative clustering (e.g., GMM but not VQVAE can yield disentangled modes).

A general implication is that discrete, unsupervised primitives discovered in this way provide a practical basis for high-level control and composable planning in robotics, without reliance on expert-annotated data.

7. Future Directions and Challenges

Several open avenues and limitations persist:

Coverage and Expressivity: Scalability to high-dimension environments, richer or hierarchical compositionality (e.g., curriculum learning, stacking discovered options/skills), and balancing coverage of rare and frequent factors remain central (Hosseini et al., 2 Feb 2026, Kim et al., 2023).
Difficulty-Aware and Goal-Conditioned Discovery: The integration of controllability measures, curriculum mechanisms, and goal-conditioned planning with unsupervised skill discovery is under active exploration (Park et al., 2023, Kim et al., 2023).
Robustness and Transfer: Robust unsupervised discovery in the presence of noise, partial observability, or complex sensorimotor contingencies, and the transferability of skills to novel tasks or domains, especially in real-world hardware deployments, continue to challenge current methods (Wang et al., 2024, Atanassov et al., 2024).
Symbolic-Subsymbolic Bridging: Extending symbolic schema discovery to handle raw perceptual input (e.g., via representation learning linking visual features to predicates) is a major open question (Suárez-Hernández et al., 2020).
Open-World and Continual Discovery: Future approaches are investigating continual, open-world action discovery with streaming data and the potential for continual integration of new actions as agents encounter novel domains (Spurio et al., 7 Aug 2025).

Unsupervised action discovery remains a foundational problem at the intersection of RL, perception, video understanding, and planning, with ongoing advances in factorization, information-theoretic objectives, and cross-modal representation learning accelerating empirical progress.